The Dumb Reason Your Fancy Computer Vision App Isn't Working: Exif Orientation (medium.com) 64
Adam Geitgey: Exif metadata is not a native part of the Jpeg file format. It was an afterthought taken from the TIFF file format and tacked onto the Jpeg file format much later. This maintained backwards compatibility with old image viewers, but it meant that some programs never bothered to parse Exif data. Most Python libraries for working with image data like numpy, scipy, TensorFlow, Keras, etc, think of themselves as scientific tools for serious people who work with generic arrays of data. They don't concern themselves with consumer-level problems like automatic image rotation -- even though basically every image in the world captured with a modern camera needs it. This means that when you load an image with almost any Python library, you get the original, unrotated image data. And guess what happens when you try to feed a sideways or upside-down image into a face detection or object detection model? The detector fails because you gave it bad data. You might think this problem is limited to Python scripts written by beginners and students, but that's not the case! Even Google's flagship Vision API demo doesn't handle Exif orientation correctly. And while Google Vision still manages to detect some of the animals in the sideways image, it detects them with a non-specific "Animal" label. This is because it is a lot harder for a model to detect a sideways goose than an upright goose.