HmmMaybe wraparound? Depending on how the values are handled, a value of 258 can end up as 2 (258-255). Hard to say without getting into the guts of the code.
(,
Tue 31 Jan 2012, 23:07,
archived)
Yes, I was thinking that, integer overflowexcept there's no good reason for an overflow to occur at that boundary between 172 and 173, unless there's something scaling the 172 up to some 2^N-1, and though I fiddled around with the calculator for ages I couldn't find any reasonable way for that to happen. To get the 255/256 boundary you have to scale them both up by around 1.48; so that doesn't seem to mean anything. If you multiply by three, they just miss the 511/512 boundary: 172*3 is 516. Maybe *3 and then minus 5? Why, though?
(,
Tue 31 Jan 2012, 23:53,
archived)
This is getting far too geekyThe only thing I could think of was bad handling of signed/signed bytes. Then you get wraparound/overflow at 127. Java did it to me a lot when I ported a bunch of my old c/c++ code (the bastard, *shakes fist*).
[edit] Using the luminance algorithm, 173 * 0.72 is 124(ish), which is pretty close to the boundary.
[editedit] Except 0.72 is the factor for the green channel. Fucksocks!
(,
Wed 1 Feb 2012, 0:40,
archived)
[edit] Using the luminance algorithm, 173 * 0.72 is 124(ish), which is pretty close to the boundary.
[editedit] Except 0.72 is the factor for the green channel. Fucksocks!