I have added a new experiment called Galactose to my Incubator.
The motion of a particle field is controlled by the shape of the random clusters they form among each other. The orientation of the clusters control each particle’s further motion which leads to all kind of interesting feedback effects and emergent patterns. The core algorithm which calculates the overall alignment and orientation of the clusters is written in Pixel Bender.
Let me say this first – I am not a fan of Canvas and tinkering around with this project I realized why I prefer programming in AS3 so much more to Javascript: my goodness – this language is a dirty mess, how can anyone get anything done with it? But for fairness sake – I guess there must be some pro development tools out there which I don’t know about and my using of notepad might not be the true JS developer experience.
So even though I think that compared to the capabilities of the Flash Player Canvas still lacks a lot, it cannot be ignored since it is now available on most major browsers. As a Flasher I find it especially entertaining to watch how history keeps repeating – browsing Canvas community blogs is like time traveling: plasma effects, l-systems, bitmap manipulations, physics engines, vector drawing experiments, emulators – in short everything that we were excited and enthusiastic about years ago is now being rediscovered, ported or reinvented on Canvas, just like we rediscovered, ported or reinvented things that had been done in Java or C years before Flash. But hey, I actually envy these guys – there is nothing more rewarding and creativity boosting than trying to push a limited platform to the maximum.
So since we are not so different after all, here’s a little contribution to world platform peace – Pibeca allows you to use pixel shaders on Canvas Bitmaps. It achieves that by using Adobe® Pixel Bender™ filters which can be written with the freely available Pixel Bender™ toolkit. Those kernels are very small programs who’s only purpose in life it is to push pixels around very fast.
So here is an proof of concept which should work on any browser that supports canvas:
Under the hood a canvas bitmap and a kernel url is being sent to an invisible SWF which loads the Pixel Bender filter kernel, processes the bitmap and then sends it back to JavaScript which passes it back to a canvas. That’s all. Given that I have to encode the bitmap to a string in order to get it to the other side and back the speed is pretty okay I think.
I wanted to get this prototype out quickly, so until I have prepared some docs and more examples those who are adventurous can already have a look at QuasimondoLibs and check out the Actionscript and Javascript source code for this demo. Pibeca is released under MIT License.
Sometimes when working with bitmaps you might find the need to work in a HSL (Hue, Saturation, Luminance) mode, for example when trying to detect skintones in an image. So what you do is probably to use the classic formula from Wikipedia that uses min/max and a whole lot of ifs: http://en.wikipedia.org/wiki/HSL_and_HSV
Well, here is a little alternative method: whilst working with the YUV colorspace I figured that since Y contains all the luminance information, the U + V channels must thus contain the hue and saturation. And it turns out that indeed when looking at u and v as the coordinates of a vector, its angle will represent the hue and its length will be the saturation.
So a formula that does not need any min/max and ifs to convert rgb to hsl looks like this:
// r,b and b are assumed to be in the range 0...1 luminance = r * 0.299 + g * 0.587 + b * 0.114; u = - r * 0.1471376975169300226 - g * 0.2888623024830699774 + b * 0.436; v = r * 0.615 - g * 0.514985734664764622 - b * 0.100014265335235378; hue = atan( v, u ); saturation = Math.sqrt( u*u + v*v );
In this case hue will be between -Pi and Pi and saturation will be between 0 and 1/sqrt(2), so you might want to multiply the saturation by sqrt(2) to get it in a range between 0 and 1.
Of course this also works the other way round – hsl to rgb looks like this:
// hue is an angle in radians (-Pi...Pi) // for saturation the range 0...1/sqrt(2) equals 0% ... 100% // luminance is in the range 0...1 u = cos( hue ) * saturation; v = sin( hue ) * saturation; r = luminance + 1.139837398373983740 * v; g = luminance - 0.3946517043589703515 * u - 0.5805986066674976801 * v; b = luminance + 2.03211091743119266 * u;
And here is a demo of a Pixel Bender kernel that allows you to adjust hue, saturation and brightness of a bitmap using this technique: HSL Adjust Demo, Source: hsladjust.zip
[Edit]As it was correctly noticed in the comments, the values that this method returns are not the HSL values that you get when you are using the classic formula. Nevertheless I think that this method has its uses, for example if you want to quickly create color schemes or if you are using it like in the demo to change the hue/saturation/Luminance of a photo.[/Edit]
[Edit]I updated the factors in the matrix one more time with values I found in the Wikipedia YUV discussion page. Those look better to me at least.[edit]
Given my track record of last year – a total of 4 blog posts – I do not have very high hopes that I’ll do much better in 2011, but since I decided not to speak on any conferences this year I might have a little bit more time to share my experiments here instead of on stages only. I don’t want to over-promise, but I hope that over the next weeks I’ll find the time to finally explain and release the source codes for several projects that I have exclusively shown in my talks over the last years. Of course some of them have in the time since I’ve presented them been independently discovered by others, but I think I’ve still got a few aces in my pockets which are worth sharing.
I’ll start with a simple visual effect: Radial Blur, which can give images the 200mph speed zoom look. In a naive approach you could achieve this effect by replacing every pixel with a weighted average of all neighboring pixels within the blur radius that lie on a line that goes through the center of zoom and the pixel itself. The problem with this method is that it is very slow since it would require many calculations to collect each neighbor and since the angle is different at every pixel there is not much room for other optimizations, like reusing already read pixels, which is key for every fast blur algorithm.
But using a neat trick that I remembered from my early Photoshop days allows to speed up this process a lot. Here is a demo:
The way this works is by converting the pixels from cartesian mapping (where the axes are x & y) to polar mapping (with axes angle & distance to the center) which arranges them in a way that all the neighbors that need to be averaged arrange themselves nicely in a straight line. This allows us to use a simple native Blur filter to quickly calculate the average. Depending if the blur is in x direction or y direction the final result will either be a Radial Blur (the well-known zoom effect) or a Circular Blur (which looks like a fast spinning wheel). The final step is to convert the pixels back from polar mapping to cartesian.
Open this link in a new window to see the process in a step by step demo.
By default the mapping uses the center of the image as the center of the polar coordinate system. This will result the center of the zoom to also be in the center of the image. So what if we want the zoom center to be at a different location? There are two ways of going about this – unfortunately I did not manage the faster way to work correctly yet which would be to simply use that center in the mapping algorithm. For some reason I have not found the right equations which work both ways. So for now I am using the slower method which does always work:
The trick here is to first make the image bigger by padding the borders so that the desired center of the zoom is temporarily the center of the image. So if you want the zoom center to be in the upper half of the image you add as many border pixels to the top that the chosen spot becomes the new vertical center. Then follows the cartesian-to-polar transform, the linar blur and the polar-to-cartesian mapping. The final step is to crop the image back to its original size by removing the previously added top pixels.
Another issue are artifacts that show due to the mapping. The problem is that since all the pixels that lie on a circle with a certain radius are mapped to a straight line, once the circumference of that circle gets bigger than the width of the mapped image pixel information gets lost. To fix this it is required to make the bitmap that holds the polar map as wide as the biggest possible circumference. Unfortunately that is not always possible since the temporary bitmaps could get pretty huge this way. The good news is that since we are blurring those artifacts will mostly not become too visible. Only for small blurs it improves the quality when using a bigger resolution for the cartesian image map.
You can find the source code for the RadialBlur class and the required pixel bender kernels as usual inside QuasimondoLibs
Whodini, a popular 1980s rap group, will perform at 7 p.m. Oct. 8 at the Sasser Flea Market, 8109 Albany Highway.
