I'm all for academic exercises for the benefit of learning. I think you will find this page of some help in continuing to explore the jpeg and jfif standards. However if your goal is simply to verify if an image is valid or not, that is problematic, because jfif allows for sections of a file to be ignored, so that special data could be placed there when the file is created. You could look at exif as essentially being this type of extension, so using the exif check functions is valuable in combination with other techniques. Exif data doesn't have to be there, but if you decide that you will only accept images that also have exif, then that is another valuable and efficient check, as you can use an exif checking function to exclude images that don't have valid exif data.
In general, the proven method of knocking down malicious images is to use a combination of getimagesize and imagecreatefromstring, or the imagemagick routines kicken referenced. You used getimagesize to knock down files you have already decided are too large, and then recreate the image from file data. Either of these failing should cause rejection.
Trying to go through the files and decipher them is most certainly a block operation where you would want to read the binary values, looking for the segments, and have routines that can decipher those individual segments. A simple loop is not going to be maintainable in my opinion.
If I was trying to do this, I'd also want to try and see what gd and/or imagemagick source is doing, as those are both open source libraries written in c/c++.
For example, imagemagick has a component used to identify the internals of an image. It's available in their command line tool that allows analysis and modification of an image.
The source is here.
A very large and complicated bit of code it seems.