Most AMR perception debates start as "LiDAR or camera?" and end with a robot carrying both. That outcome makes sense once you look at what each sensor measures and where it goes blind. A LiDAR returns precise range along a limited set of beams, while an RGB-D camera returns a dense depth image with color attached but over a shorter, narrower volume. Orbbec's engineering overview of how LiDAR and depth cameras complement each other in robots lays out that split and the fusion approaches that follow from it. The harder question for a system designer is which sensor should own which job on a specific robot and when a second sensor earns its cost, power, and integration time.
What each sensor actually measures
LiDAR
A mobile robot LiDAR fires laser pulses and times their return. In direct time-of-flight (dToF) designs, each return is an independent range measurement, so error stays fairly flat with distance compared with triangulation methods. The output is a point cloud sampled along the scan pattern. A single-line 2D LiDAR covers one horizontal plane. A 3D LiDAR adds vertical coverage through multiple channels, a MEMS mirror, or a combination of scanning mechanisms.
The defining trait is sparsity. Angular spacing between beams is fixed, so the gap between neighboring points grows with range. A wall at 30m produces a clean line. A table leg or a dangling strap at the same distance may fall between beams entirely.
Orbbec's own LiDAR line shows the spread of designs. The Pulsar SL450 is a single-line dToF unit with a 270° horizontal scan. The Pulsar ME450 is a 3D dToF LiDAR that combines MEMS pitch scanning with motorized azimuth control, emits 200 kHz laser pulses, and reaches up to 45m at 90% reflectivity. It can run repetitive or non-repetitive scan patterns, which changes how evenly points fill the field of view over time.
RGB-D depth cameras
A depth camera produces a range value for nearly every pixel, registered to a color image. The depth comes from stereo triangulation, structured light, or indirect time-of-flight, and each method has its own error profile. Stereo error grows roughly with the square of distance and shrinks as baseline increases. That's why long-baseline stereo cameras publish accuracy at specific ranges. The Gemini 335L, with a 95mm baseline, is rated at ≤0.8% at 2m and ≤1.6% at 4m.
The defining trait here is density. A camera at 1280 × 800 delivers about a million depth points per frame across its field of view, so small, low, or irregular objects close to the robot show up as shapes rather than as a few scattered returns. The color channel adds semantics. A pallet, a person, and a cardboard box at the same range look identical to a range sensor but very different to a classifier running on aligned RGB.
Where the differences show up on a real robot
Range and coverage pull in opposite directions. LiDAR sees far and wide, which is what localization needs. Depth cameras see close and in detail, which is what local obstacle handling and manipulation need. The useful mental model is that LiDAR answers "where am I in this building?" while the depth camera answers "what exactly is in front of my bumper?"
Vertical coverage is where 2D LiDAR fails most often in practice. A scan plane mounted at 20cm misses table tops, shelf lips, forklift tines, and overhanging pallet loads. It also misses floor-level hazards below the plane, such as cables, dropped packaging, and dock edges. A forward-facing depth camera covers that volume directly, and its points can be inserted into a 3D costmap so the planner treats an overhang as an obstacle.
Lighting affects the two sensors differently, and the details matter more than the general claim. dToF LiDAR uses narrow-band filtering and short pulses, so ambient light has limited effect on it. Among depth cameras, structured light and indirect ToF are the most sensitive to sunlight because they depend on a projected or modulated signal. Active stereo cameras that run active and passive stereo simultaneously, such as the Gemini 330 series, keep matching on natural scene texture when sunlight washes out the projected pattern. That makes them viable near loading dock doors and skylights where other depth methods degrade.
Both sensor types have trouble with glass, mirrors, and polished floors. LiDAR beams pass through glass or reflect off it. Depth cameras see reflections as phantom geometry. Neither sensor solves this alone, and fused stacks usually add map annotations or keep-out zones for known glass partitions.
Data volume is the last difference. A 2D LiDAR scan is a few hundred to a few thousand points. A depth stream at 30fps is tens of millions of points per second. Cameras that compute depth and depth-to-color alignment on an onboard ASIC, such as Orbbec's MX6800 in the Gemini 330 series, hand the host a finished depth image rather than raw stereo pairs. The host still needs to downsample and filter before costmap insertion, so plan your voxel resolution and region of interest before you plan your compute budget.
Robot types where one sensor dominates
LiDAR-led platforms
Large-area logistics robots such as tuggers, pallet movers, and autonomous forklifts depend on LiDAR. They navigate long aisles at speed, need stable localization against walls and racking, and typically carry safety-rated laser scanners for personnel protection under standards such as ISO 3691-4. Perception cameras on these platforms supplement the safety function rather than replace it. Outdoor yard and campus robots also lean on 3D LiDAR for mapping at range, often paired with stereo cameras for terrain and near-field obstacles.
Camera-led platforms
Indoor service robots, such as cleaning machines, hotel and restaurant delivery robots, and hospital carts, often lead with depth cameras. Their speeds are low, their environments are cluttered with furniture and people, and the obstacles that matter are close and irregular. Visual SLAM with RGB-D input can carry localization in feature-rich indoor spaces, and the camera's color stream supports person detection and semantic navigation. Many of these robots still add a low-cost 2D LiDAR for localization robustness in long, featureless corridors.
Mobile manipulators belong in this group too, at least at the arm. A camera on the base handles navigation and approach. A separate compact camera at the wrist handles the grasp. For arm-mounted picking, the working distances are short. The Gemini 305 has a 4cm minimum working distance and an ideal range of 7 to 50cm, which matches the last half-meter of a pick far better than a navigation camera tuned for meters.
The fused stack on a warehouse AMR
A typical goods-to-person or case-picking AMR divides the work by range and by question.
LiDAR owns global localization. A 2D or 3D scan matched against a prebuilt map gives a stable pose estimate in aisles where the geometry is consistent. ROS 2 teams commonly use packages such as slam toolbox for mapping and AMCL or a scan matcher for localization at runtime.
A forward-facing stereo depth camera owns the near field. Its points feed a voxel or obstacle layer in the local costmap, catching overhangs, low debris, and partially loaded carts that the scan plane misses. In Nav2, that typically means a LiDAR-fed obstacle layer and a camera-fed voxel layer feeding the same local costmap, each with its own clearing and marking ranges.
A wrist camera owns the pick. On robots with an arm, a short-range camera such as the Gemini 305 localizes the item in the tote or on the shelf and guides the end effector through the final approach.
The fused stack also covers each sensor's failure modes. When a forklift parks in a mapped aisle and changes the geometry, the depth camera still sees what is directly ahead. When a dark, low-texture surface weakens camera depth, the LiDAR still holds the robot's pose.
Making fusion work in practice
Most fusion problems on real robots trace back to calibration and timing, not algorithms.
Extrinsic calibration between the LiDAR and each camera has to be measured, not read from CAD. A small angular error in a camera mount becomes a large position error several meters out. Mount the sensors rigidly to the same structure and recheck calibration after any service work that touches the brackets.
Time synchronization matters as soon as the robot moves. A 30ms offset between a LiDAR scan and a camera frame at 1.5m/s is 45mm of disagreement, enough to smear obstacle edges in the costmap. Hardware triggering, hardware timestamps, and a shared clock across sensors reduce this. Gemini 330 series cameras support hardware timestamping and multi-device synchronization, and Orbbec's SDK supports both its Pulsar LiDARs and its depth cameras, which keeps driver handling in one framework for teams that run both.
State estimation ties it together. An extended Kalman filter or factor-graph estimator fusing wheel odometry, IMU data, and LiDAR scan matching is standard for pose. Depth cameras usually feed the map and costmap rather than the pose estimate, unless the platform relies on visual odometry. Decide early which sensor is authoritative for which quantity. Ambiguity there shows up later as oscillating behavior near obstacles.
Questions to settle before you pick sensors
Start with the robot's speed and stopping distance, since that sets how far ahead you need reliable detection. Then map the obstacle classes the robot will actually meet. If the hazards are mostly walls and racking, LiDAR alone may carry the job. If they include overhangs, people, low debris, or items the robot must pick up, a depth camera is close to mandatory.
Next, look at the environment. Long, repetitive aisles favor LiDAR localization. Cluttered, feature-rich indoor spaces support visual approaches. Sunlit docks and outdoor transitions narrow the depth camera choice toward active stereo.
Finally, count the costs that don't appear on the purchase order. These include host compute for dense depth, cable routing and interface type, calibration fixtures, and the engineering hours spent on synchronization. A second sensor that the team can't calibrate and maintain in the field adds risk rather than removing it.
Frequently asked questions
Can a depth camera replace LiDAR on an AMR?
On small indoor robots moving at low speed, often yes. RGB-D visual SLAM works well in feature-rich spaces, and the camera already provides the near-field obstacle data. On larger robots that travel long, repetitive aisles at higher speed, depth cameras alone struggle with localization because their useful range is shorter and the scene repeats. Most of those platforms keep at least a 2D LiDAR for pose.
Why add a depth camera if the robot already has a 3D LiDAR?
A 3D LiDAR fixes the vertical coverage problem of a 2D scanner, but its points are still sparse at the distances where close obstacle handling happens, and it carries no color. A depth camera adds dense geometry within a few meters and aligned RGB for classification. If the robot needs to tell a person from a pallet or pick an item, the camera does work the LiDAR can't.
What usually goes wrong when teams fuse LiDAR and camera data?
Calibration drift and timing offsets cause most problems. If the extrinsics between sensors are off by a degree or the timestamps disagree by tens of milliseconds, the same obstacle appears in two places in the costmap, and the planner hesitates or swerves. Rigid shared mounting, hardware synchronization, and a calibration check in the maintenance routine prevent most of it.
