With iOS 5 core image filters were added to the iOS 5 sdk providing a high level API for filtering images, and video.
Unfortunately, left out on is the key ability to create completely custom image filters. You can however do this using OpenGL ES directly, but this requires a ton of boiler plate code, and is a pretty complicated process.
fortunately Brad Larson who created the popular open source app Molecules which is great for learning OpenGL ES has developed a library making it considerably easier to create high performance custom image filters using OpenGL ES 2.0.
The library is GPUImage includes several example video filters so you can see exactly how it’s done.
You can find the library on its Github repository here.
You can find out more about the library, it’s architecture, and how to use it on Brad’s site here.
I’ve received quite a few inquiries about how to create custom video filters using Core Image, and while that can’t be done – using the GPUImage framework provides a nice head start doing so with OpenGL ES 2.0