Capturing High Resolution Photos

Episode #303 | 17 minutes | published on September 25, 2017 | Uses Xcode-9.0-beta6, swift-4

Subscribers Only

I got a tip from a subscriber about the method I used to capture the photo in the last episode. Since we're capturing the data using the preset we chose for processing the rectangles, we are bound to that preset when we export to an actual photo. By using AVCapturePhotoOutput, we can have much more control over the format, size, and quality of the resulting image. In addition, since we are leveraging the SDK for capturing the photo, we benefit from things like auto-flash, HDR, auto focus, and the built-in camera shutter sound. (Yay for deleting code!) The end result might not look very different on device, but if you are taking that image somewhere else to do OCR or other processing on it, a higher quality image is important.

This episode is part of a series: Camera Capture and Detection.

1. Intro 1 min

2. Setting Up The Project 9 min

3. Camera Preview and Sample Buffer 10 min

4. Drawing on Top of the Video Preview 14 min

5. Fixing the Coordinates 7 min

6. Perspective Outlining 9 min

7. Camera Sounds and UI 12 min

8. Rendering the Final Image 15 min

9. Capturing High Resolution Photos 17 min

10. Adding a Flash Toggle Camera Control 12 min

Episode Links

AVCapturePhotoOutput Reference

Setting Up

First, we'll add a property to store our photo output instance:

var photoOutput: AVCapturePhotoOutput?

Then, when we are setting up our capture session, we'll also add this output to it:

            let photoOutput = AVCapturePhotoOutput()
            self.photoOutput = photoOutput
            captureSession.addOutput(photoOutput)

Taking the Photo

Later on, when we want to take a photo, we no longer need the wantsPhoto flag. Instead, we can just tell the output to capture a photo. This requires us to tell the AVCapturePhotoOutput what pixel format we want to use (among other things).

If we don't do this we'll get a processed JPEG image, and this would need to get converted back to an uncompressed type for us to use Core Image on.

func takePhoto() {
    guard let formats = photoOutput?.supportedPhotoPixelFormatTypes(for: .tif) else { return }
    print("available pixel formats: \(formats)")
    guard let uncompressedPixelType = formats.first else {
        print("No pixel format types available")
        return
    }

    let settings = AVCapturePhotoSettings(format: [
        kCVPixelBufferPixelFormatTypeKey as String : uncompressedPixelType
    ])
    settings.flashMode = .auto

    photoOutput?.capturePhoto(with: settings, delegate: self)
}

The formats returned by the method above are of type OSType. these are special integers that have a nifty attribute: they can be printed as text!

If you want to print out the actual 4-letter code for a given OSType, you can use this function:

    func str4 (_ n: Int) -> String {
        var s: String = String (describing: UnicodeScalar((n >> 24) & 255)!)
        s.append(String(describing: UnicodeScalar((n >> 16) & 255)!))
        s.append(String(describing: UnicodeScalar((n >> 8) & 255)!))
        s.append(String(describing: UnicodeScalar(n & 255)!))
        return s
    }

Capturing the Raw Pixels

This delegate method will be called once the photo has been taken and pixel data is available. We want to hand this data off to core image.

extension CaptureViewController : AVCapturePhotoCaptureDelegate {
    func photoOutput(_ output: AVCapturePhotoOutput,
                     didFinishProcessingPhoto photo: AVCapturePhoto,
                     error: Error?) {

        guard let rect = lastDetectedRectangle else { return }
        imageProcessingQueue.async {
            guard let pixelBuffer = photo.pixelBuffer else {
                print("No pixel buffer provided. Settings may missing pixel format")
                return
            }

            let ciimage = CIImage(cvPixelBuffer: pixelBuffer)
            if let correctedImage = self.perspectiveCorrect(ciimage, rectFeature: rect) {
                DispatchQueue.main.async {
                    self.displayImage(ciImage: correctedImage)
                }
            }
        }
    }
}

Cleaning Up

Since we're using the built-in camera features, we can delete the code that flashes the screen and plays the camera shutter sound. It's not needed anymore!

View on Github

Download