<p>(Crossposting from here: <a href="https://softwareengineering.stackexchange.com/questions/399331/fuse-3d-points-in-bundel-adjustment">https://softwareengineering.stackexchange.com/questions/399331/fuse-3d-points-in-bundel-adjustment</a>)</p>

<p>I'm actually implementing my own Pose-Estimation/- and -Refinement pipeline. For this purpose I use one moving mono-camera. Then I take the consecutive images to estimate the pose and triangulate the points (nothing special). In the last step I refine the poses and 3D-points with a bundle adjustment approach. </p>

<p>Generating 3D points with triangulation from consecutive image pairs will give me multiple estimations for one real-world 3D point. In fact, all the estimations refer to the same point. For my understanding, these estimations of the same 3D-points have to be fused in some way. Otherwise the poses were not linked anymore through a common point (see also image below). Further, looking at the equation for the re-projection error in different publications:</p>

<p><a href="https://i.sstatic.net/5Bmdqm.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/5Bmdqm.png" alt="Core equation Bundle Adjustment"></a></p>

<p>turns out, that 3D point (vector a) is only related to <strong>j</strong> and not to the cameraindex <strong>i</strong>.</p>

<p><a href="https://i.sstatic.net/hl3QAm.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/hl3QAm.png" alt="Stolen from: SBA: A Software Package for Generic Sparse Bundle Adjustment, MANOLIS I. A. LOURAKIS and ANTONIS A. ARGYROS"></a></p>

<p>Do I understand that right or do I have to use a different set of 3D points for each camera view? Suppose I've to merge the 3D points, is there any preferable strategy?</p>

<p>Thanks in advance!</p>

<p>I know, there are already countless implementations for BA. I want to use it for further development...</p>

CC BY-SA 4.0
2019-10-11T15:28:37.487Z
I'm voting to close this question as off-topic because it was cross posted on another site and answered there already.
Catija
CC BY-SA 4.0
2019-10-12T22:01:27.337Z
So, everything works now as expected and the results seem to be correct. I took the average of all 3D-Point belonging to one physical point. Therefore this question should be considered as answered.

Just a point for further investigation: Taking the average may not be a really robust way. It would be useful to implement something like an outlier control. But this was not part of the question at core.

Please consider this as the answer. I can't mark it as an answer anymore...
MattDom
<image-processing><optimization>
Fuse 3D-Points in Bundle Adjustment?
MattDom
# Fuse 3D-Points in Bundle Adjustment?

(Asked by: username_0 on 2019-10-09)

(Crossposting from here: <https://softwareengineering.stackexchange.com/questions/399331/fuse-3d-points-in-bundel-adjustment>)

I'm actually implementing my own Pose-Estimation/- and -Refinement pipeline. For this purpose I use one moving mono-camera. Then I take the consecutive images to estimate the pose and triangulate the points (nothing special). In the last step I refine the poses and 3D-points with a bundle adjustment approach.

Generating 3D points with triangulation from consecutive image pairs will give me multiple estimations for one real-world 3D point. In fact, all the estimations refer to the same point. For my understanding, these estimations of the same 3D-points have to be fused in some way. Otherwise the poses were not linked anymore through a common point (see also image below). Further, looking at the equation for the re-projection error in different publications:

[![Core equation Bundle Adjustment](https://i.sstatic.net/5Bmdqm.png)](https://i.sstatic.net/5Bmdqm.png)

turns out, that 3D point (vector a) is only related to **j** and not to the cameraindex **i**.

[![Stolen from: SBA: A Software Package for Generic Sparse Bundle Adjustment, MANOLIS I. A. LOURAKIS and ANTONIS A. ARGYROS](https://i.sstatic.net/hl3QAm.png)](https://i.sstatic.net/hl3QAm.png)

Do I understand that right or do I have to use a different set of 3D points for each camera view? Suppose I've to merge the 3D points, is there any preferable strategy?

Thanks in advance!

I know, there are already countless implementations for BA. I want to use it for further development...

**Tags:** image-processing, optimization

> **Comments:**
> **username_1**: I'm voting to close this question as off-topic because it was cross posted on another site and answered there already.  
> **username_0**: So, everything works now as expected and the results seem to be correct. I took the average of all 3D-Point belonging to one physical point. Therefore this question should be considered as answered.  Just a point for further investigation: Taking the average may not be a really robust way. It would be useful to implement something like an outlier control. But this was not part of the question at core.  Please consider this as the answer. I can't mark it as an answer anymore...  

---

## 0 answers

