Photogrammetry 101 using Deep Trekker ROVs

Introduction

What is this guide and who is it for?

This is a basic tutorial for users who want to generate underwater 3D models with Deep Trekker ROVs. It is targeted for beginners to photogrammetry, and will highlight the key considerations for data capture and show step-by-step instructions on how to create and export a 3D model. 

What is photogrammetry?

Photogrammetry is the science of extracting 3D information from photographs. In the context of underwater photogrammetry with Deep Trekker ROVs - the idea is to capture a large set of images of an underwater asset (object, structure, etc) from different angles and positions, then use a software application to generate a photogrammetric 3D model.

Background & Preparation

Bridge Controller Setup

The remainder of the guide is written with the assumption that the end-user has a Deep Trekker ROV equipped with a 4K UHD Camera. The Auto-Snapshot feature will only work with this camera.

Software Version

Update the software of the entire Deep Trekker system to ensure you have all the latest updates and features unlocked. For photogrammetry, versions 4K UHD Camera v3.1+ and Bridge Controller v8.6+ are strongly recommended.

Auto-Snapshot (Timed Snapshot) Feature

Users wanting to create 3D models, must use the Auto-Snapshot feature - easily accessible through the Camera Settings Page or by assigning it to a Programmable Button. This feature captures high-resolution images at a preset interval, with a recommended rate of 1 image per second to maximize image coverage. When activated, it automatically adjusts certain camera settings to optimize for photogrammetry. After the feature is turned off, users will find a date-time-stamped folder on their SD card containing all the images from the session.

White Balance

It is recommended to use the default Auto White Balance mode to ensure the camera utilizes Deep Trekker’s underwater-tuned color correction algorithm. Please note that this feature requires 4K UHD Camera v3.1+ and Bridge Controller v8.6+. Advanced users may opt for manual white balancing if desired.

Lighting

To correctly illuminate the object in the absence of natural light, strong auxiliary lighting is required. It is recommended to use the auxiliary lights on the ROV only - IE. do not use the main camera head lights. The reason for this is to minimize backscatter (the illumination of particulates) in front of the ROV.

Scaled Model

Normally when generating a model, the model's dimensions have no correlation to a real-world size. To achieve 3D modeling with real-world scale, there needs to be a known length in the 3D scene, such as the diameter of a pipe, or the length of some part of a structure. If no lengths are known, an underwater scale-bar (or ruler) should be deployed into the scene to be modeled. With this information, the software (3DF Zephyr) can scale the entire 3D model to real-world units. The measurement accuracy of the 3D model is dependent on many factors, thus it will not be covered here.

Environmental Conditions

Photogrammetry performs best in clear, non-turbid conditions, where the appearance of the object remains consistent at varying distances. In non-ideal conditions, maintaining a closer fixed distance to the object can help ensure that textured details are visible. Moving too far away may result in hazy images due to particulate interference.

Inspection Tips

Much like photography, underwater photogrammetry is an art form. Even with considerable expertise, unfavorable environmental conditions can make it quite challenging to generate a model. Often, the challenges in generating 3D models arise not from the software or equipment but from various factors related to image data acquisition such as  environmental conditions, lighting, image overlap, and more. Understanding and managing these variables is key to producing superior 3D models.

The following are the most important tips to remember when planning a photogrammetry inspection with an ROV.

Object under Inspection

• The object under inspection should not be moving whatsoever. Any biofouling moving on the object will cause problems for model generation! Similarly, avoid stirring up any dirt or debris from the seafloor. 

• The object under inspection should have many distinguishable features, such as edges, marks, wording, or textured details. Plain surfaces, like bare walls, are more challenging to model.

Pre-dive checks

• Ensure the external viewport of the camera is free of debris or dirt

• Remove grabber if it is always in view of the camera

While Piloting

• Ensure the grabber, tether or any other part of the ROV is not in view during the inspection.

• Ensure consistent lighting during the inspection. Do not turn on/off the lights during the capture period.

• Drive slowly! Reduce the ROV’s speed and steer gains to minimize undesired motion blur and maximize overlap between photos.

• During capture, focus on the object under inspection, and avoid looking away or looking at things other than the object

After Piloting

• After stopping the Auto-Snapshot feature, use the Controller’s File Manager to review that the images have been captured as desired.

Driving Style

While there is no single correct method to drive the ROV, it is often helpful to follow a logical path or pattern while capturing data. The figures below demonstrate some of the options. The key takeaway is to maximize overlap between images, moving both laterally and vertically, and capturing the object from all directions and angles.

Ideal Capture Example

• Observe slow, panning motion of ROV

• No foreign objects aside from the object of interest

• Observe overlap between consecutive photos

Common Mistakes & Environments to Avoid

• Avoid shallow scenes where surface ripples cause dynamic light patterns underwater

• Grabber should not be visible

  • Remove or adjust camera head (depending on ROV)

• Avoid stirring up debris from sea floors and surfaces

• Avoid inspecting objects with moving parts - biofouling in this case

• Remember to drive as slowly as possible to minimize object motion caused by thruster induced water currents.

• Avoid capturing water’s surface reflection

• Avoid capturing the tether in the scene

• Avoid looking away from object under inspection

• Drive and steer the ROV slower than shown here

• Do not rotate in place when possible.

  • Maximize the use of lateral and vertical motion instead

Generating the 3D Model

Prerequisites

Download and install a version of 3DF Zephyr on an up-to-date, capable Windows computer. The recommended specifications for processing can be found here. 3DF Zephyr is a software that will take the image dataset and generate the 3D model.

For users new to photogrammetry, we recommend getting the 30-day trial of the full version, or purchasing the Lite version. The full comparison of software license types is here. 

Step-by-Step Instructions

The following will walk-through how to generate a 3D model, using an example dataset found here. It is the model of a submerged canoe captured with a Deep Trekker Revolution ROV. 

• Copy the captured image dataset from the SD card to the PC with 3DF Zephyr installed.

• Open 3DF Zephyr and from the top toolbar, click Workflow -> New Project

• In the Project Wizard dialog, click on the Texture tile and then click Next

• In the Photo section stage, click the plus (+) tile to add the photos.

• In the file browser dialog, browse to the location of the image dataset, and select all photos in the folder. Then click Open

• You should now see a list of all the photos to process. Click Next

• In the Camera Calibration dialog, let the software auto-calibrate the camera. Simply click Next

• The next few dialogs will select presets for each stage of processing. All default options is a good starting point, but we have found the below presets to work best: 

  • For Camera Orientation, select the “Deep” preset. 

  • For Dense Point Cloud Creation, select the “Default” preset. 

  • For Surface Reconstruction, select the “High Details” preset. 

  • For Texturing, select the “Default multi texture” preset.

• Finally, click Run to process the 3D model. Be patient - the larger the dataset, the longer it will take to process.

• Once complete, use the mouse to inspect the 3D model. By default the “Textured Mesh” is visible, but the various layers, such as dense point clouds, can also be viewed by enabling and disabling the checkboxes in the left Project pane. 

• At this point, you should save the project, by clicking File -> Save As…, and select a location to save the project.

• If you want to share the 3D model, you can click on Export -> Export Textured Mesh in the top toolbar. Then change Export format to Glb, and click Export. Alternatively, export in your file format of choice.

• For more instructions and tutorials on 3DF Zephyr, in the top toolbar click on Help -> User Manual or 3Dflow Tutorials…