DivideTiles - Tile Processing

Overview

The DivideTiles interface is used to divide scenes into tiles before reconstruction. This interface supports 2D regular grid, 3D regular grid, and memory-based adaptive tiling methods, suitable for processing large-scale datasets.

Use Cases

Ultra-large scale datasets (thousands of images)
Memory-limited processing environments
Projects requiring parallel processing
Distributed computing scenarios

How It Works

Interface Call

Command Line Call

# Method 1: Through main engine
reconstruct_full_engine.exe -reconstruct_type 4 -task_json divide_config.json

# Method 2: Standalone tiling engine
divide_engine.exe divide_config.json

Parameter Description

reconstruct_type: Fixed as 4 (indicates DivideTiles)
task_json: Configuration file path

Configuration Parameters

Required Parameters

Parameter	Type	Description
`working_dir`	string	Working directory (must contain AT results)
`gdal_folder`	string	GDAL data path
`resolution_level`	int	Reconstruction precision (1=High, 2=Medium, 3=Low)
`divide_parameters`	JSON	Tiling parameters
`roi_for_3d`	JSON	ROI for 3D reconstruction

Optional Parameters

Parameter	Type	Default	Description
`roi_coordinate_system`	JSON	WGS84	ROI coordinate system

Tiling Parameters (divide_parameters)

Parameter	Type	Default	Description
`divide_mode`	int	-	Tiling mode: 0=2D grid, 1=3D grid, 2=memory adaptive
`tile_size`	double	-	Grid size (required for mode 0/1)
`target_memory`	double	0	Target memory limit (GB)
`reconstruct_whole_tile`	bool	false	Whether to reconstruct complete tiles (no boundary clipping)
`tile_system`	JSON	AT coordinate system	Tiling coordinate system

Tiling Modes Explained

1. 2D Regular Grid (divide_mode = 0)

Suitable for areas with little terrain variation:

{
  "divide_mode": 0,
  "tile_size": 500.0,        // 500m x 500m grid
  "tile_system": {
    "type": 3,
    "epsg_code": 32650       // UTM projection coordinate system
  }
}

2. 3D Regular Grid (divide_mode = 1)

Suitable for areas with large elevation changes:

{
  "divide_mode": 1,
  "tile_size": 300.0,        // 300m x 300m x 300m cube
  "tile_system": {
    "type": 1                // Local coordinate system
  }
}

3. Memory Adaptive Tiling (divide_mode = 2)

Automatically tiles based on memory limits:

{
  "divide_mode": 2,
  "target_memory": 16.0,     // Maximum 16GB memory per tile
  "reconstruct_whole_tile": false
}

Complete Configuration Examples

City-Scale Scene 2D Tiling

{
  "working_dir": "C:/Projects/CityScale",
  "gdal_folder": "C:/MipMap/SDK/data",
  "resolution_level": 2,
  "divide_parameters": {
    "divide_mode": 0,
    "tile_size": 1000.0,     // 1km grid
    "tile_system": {
      "type": 3,
      "epsg_code": 32650
    },
    "reconstruct_whole_tile": true
  },
  "roi_for_3d": {
    "boundary": [
      [500000, 2500000],
      [510000, 2500000],
      [510000, 2510000],
      [500000, 2510000]
    ],
    "min_z": 0,
    "max_z": 500
  },
  "roi_coordinate_system": {
    "type": 3,
    "epsg_code": 32650
  }
}

Mountain Terrain 3D Tiling

{
  "working_dir": "C:/Projects/Mountain",
  "gdal_folder": "C:/MipMap/SDK/data",
  "resolution_level": 2,
  "divide_parameters": {
    "divide_mode": 1,
    "tile_size": 500.0,      // 500m cube
    "tile_system": {
      "type": 0,             // LocalENU
      "origin_point": [114.123, 22.123, 100.0]
    }
  },
  "roi_for_3d": {
    "boundary": [...],
    "min_z": 100,
    "max_z": 2000           // Large elevation difference
  }
}

Memory-Limited Environment

{
  "working_dir": "C:/Projects/LimitedMemory",
  "gdal_folder": "C:/MipMap/SDK/data",
  "resolution_level": 2,
  "divide_parameters": {
    "divide_mode": 2,
    "target_memory": 8.0,    // Only 8GB available memory
    "reconstruct_whole_tile": false
  },
  "roi_for_3d": {
    "boundary": [...]
  }
}

Output Results

tiles.json File Structure

The generated tiles.json after tiling:

{
  "tiles": [
    {
      "name": "tile_0_0",
      "empty": false,
      "max_memory": 12.5,    // GB
      "roi": {
        "boundary": [...],
        "min_z": 100,
        "max_z": 200
      }
    },
    {
      "name": "tile_0_1",
      "empty": false,
      "max_memory": 15.3,
      "roi": {...}
    }
  ],
  "divide_mode": 0,
  "tile_system": {...},
  "tile_grid": {             // Grid information (mode 0/1)
    "origin": [500000, 2500000],
    "rows": 10,
    "cols": 10
  },
  "tile_size": 1000.0
}

Using Tiling Results

Reconstruct3D will automatically read tiles.json and perform tiled reconstruction:

{
  "license_id": 9200,
  "working_dir": "C:/Projects/CityScale",  // Contains tiles.json
  // ... other Reconstruct3D parameters
}

Tiling Strategy Selection

Scene Type Comparison

Scene Type	Recommended Mode	Grid Size Suggestion	Description
Flat City	2D Grid	500-2000m	Flat terrain, small elevation changes
Mountain Terrain	3D Grid	300-1000m	Large elevation differences, requires vertical layering
Mixed Scene	Memory Adaptive	-	Automatically optimize tiling scheme
Linear Engineering	2D Grid	Custom	Set elongated strips along route direction

Memory Estimation

Memory requirement per tile ≈ (Images in tile × Image resolution × 3) / Compression ratio

Reference values:

100 20MP images ≈ 8-12 GB
200 20MP images ≈ 16-24 GB

Parallel Processing

Local Parallel

import subprocess
import json
import concurrent.futures

# Read tiling results
with open("tiles.json", "r") as f:
    tiles_info = json.load(f)

def process_tile(tile):
    if tile["empty"]:
        return
    
    config = {
        "license_id": 9200,
        "working_dir": f"./output/{tile['name']}",
        "tile_name": tile["name"],
        # ... other parameters
    }
    
    with open(f"{tile['name']}.json", "w") as f:
        json.dump(config, f)
    
    subprocess.run([
        "reconstruct_full_engine.exe",
        "-reconstruct_type", "2",
        "-task_json", f"{tile['name']}.json"
    ])

# Parallel processing
with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:
    executor.map(process_tile, tiles_info["tiles"])

Distributed Processing

Tiling results can be distributed to multiple machines for processing:

Master node performs tiling
Distribute tiles.json and data
Each node independently processes assigned tiles
Aggregate results

Best Practices

1. Tile Size Selection

Sufficient memory: Use larger tiles to reduce boundary effects
Limited memory: Use smaller tiles to ensure stability
Parallel processing: Balance number of tiles and processing time per tile

2. Overlap Area Handling

{
  "reconstruct_whole_tile": true,  // Generate complete tiles
  // Merge overlap areas in post-processing
}

3. Coordinate System Selection

2D tiling: Use projected coordinate system (metric units)
3D tiling: Local or LocalENU
Avoid 3D tiling in geographic coordinate systems

4. ROI Optimization

{
  "roi_for_3d": {
    "boundary": [
      // Use Smart ROI generated from AT
      // Properly extend boundaries to avoid clipping
    ]
  }
}

Performance Optimization

1. Processing Time Estimation

Total time = Tiling time + max(Processing time per tile) + Merging time

2. Load Balancing

Check max_memory distribution
Adjust parameters to balance load across tiles
Consider dynamic task allocation

3. Storage Optimization

Use SSD for working directory
Distribute tile results across multiple disks
Reserve sufficient temporary space

Common Issues

Q: Gaps at tile boundaries after tiling?

Set reconstruct_whole_tile: true
Increase overlap between tiles (by extending ROI)
Optimize merging in post-processing

Q: Some tiles fail to process?

Check if tile's max_memory exceeds limits
Review image distribution in specific tiles
Consider re-tiling or adjusting parameters

Q: Too many tiles?

Increase tile_size or target_memory
Use different tiling mode
Consider hierarchical processing strategy

Advanced Applications

Custom Tiling Schemes

Based on tiles.json, you can:

Manually adjust tile boundaries
Merge small tiles
Split large tiles
Set priorities

Incremental Updates

# Process only changed areas
changed_tiles = identify_changed_areas()
for tile in changed_tiles:
    process_tile(tile)

Next Steps

Learn about Reconstruct3D for tiled reconstruction
Check Basic Concepts for more processing techniques

Tip: A proper tiling strategy is key to processing large-scale data. It's recommended to test with small data first to find optimal parameters before processing complete datasets.

Overview​

How It Works​

Interface Call​

Command Line Call​

Parameter Description​

Configuration Parameters​

Required Parameters​

Optional Parameters​

Tiling Parameters (divide_parameters)​

Tiling Modes Explained​

1. 2D Regular Grid (divide_mode = 0)​

2. 3D Regular Grid (divide_mode = 1)​

3. Memory Adaptive Tiling (divide_mode = 2)​

Complete Configuration Examples​

City-Scale Scene 2D Tiling​

Mountain Terrain 3D Tiling​

Memory-Limited Environment​

Output Results​

tiles.json File Structure​

Using Tiling Results​

Tiling Strategy Selection​

Scene Type Comparison​

Memory Estimation​

Parallel Processing​

Local Parallel​

Distributed Processing​

Best Practices​

1. Tile Size Selection​

2. Overlap Area Handling​

3. Coordinate System Selection​

4. ROI Optimization​

Performance Optimization​

1. Processing Time Estimation​

2. Load Balancing​

3. Storage Optimization​

Common Issues​

Q: Gaps at tile boundaries after tiling?​

Q: Some tiles fail to process?​

Q: Too many tiles?​

Advanced Applications​

Custom Tiling Schemes​

Incremental Updates​

Next Steps​