<p>I'm developing a deep learning algorithm to find and classify (post consumption) plastic bottles on a conveyor belt. A way to train my model is to get tons of labeled images of each bottle, which for now is not feasible. Another possible option is to make a 3D scan of the bottle and render it in different angles thus creating synthetic training data.</p>

<p>The way I want to obtain the mesh is by using for example ReMake or CapturingReality and 2D images. The problem is that most of the bottles are transparent and are causing problems in those programs. I just ordered Chalk spray to paint them and with a marker add dots allover. Then hopefully I'd be able to render it with the specific material.</p>

<p>My question is, is there a way to add the visual details like the labels and the color of the bottle cap back to the mesh? I'll be using a RGB-D camera so the priorities are getting both the visual cues as well as the depth information as accurate as possible to what the camera will be classifying in a further stage. </p>

<p>Any advise and feedback will be highly appreciated, thanks in advanced!! </p>

CC BY-SA 3.0
2017-03-28T05:13:20.123Z
Why scan it? I mean most likely there is a model of the bottle somewhere (ask manufacturer) or then modeling it from scratch which usually is not a big deal. Also scanning is more work than taking a hundred pictures.
joojaa
CC BY-SA 3.0
2017-03-28T08:57:17.907Z
I don't think a rendering approach will work well for you in this case, as a lot of the bottles on your belt will be crushed, which is quite hard to simulate.
Dan Hulme
CC BY-SA 3.0
2017-03-28T14:17:17.487Z
@DanHulme I dont know we did it for testing a reverse vending machine for testing and it seemed to work quite well. Altough it is true that real thing has different levels of data. But it was quite much easier to have digital reolicates of 2000 bottles than actually go and buy them. as quite many of the models were available from the vendor with some emailing and asking nicely.
joojaa
CC BY-SA 3.0
2017-03-28T14:24:42.787Z
@joojaa Yeah, in general it can be a really effective technique, but if it's for post-consumption plastic bottles you can't expect them to be the same shape as new bottles
Dan Hulme
CC BY-SA 3.0
2017-03-28T14:27:44.393Z
@DanHulme Depends on where you are, in Finland where nearly all bottles and cans are recycled they will in fact all be 90% intact by the time they return as manhandling them means you will not get the deposit back. Glass bottles will be reused as bottles.
joojaa
CC BY-SA 3.0
2017-03-31T14:28:52.960Z
@joojaa Asking the manufacturer is actually the best idea. You're probably right, i was under the impression that i would need at least a couple of thousand images of each bottle. Can you tell me more about the work you did with the reverse vending machine? How did you guys go about generating the data?
Jowizo
<rendering><3d><texture><mesh>
Advice on synthetic data generation using 3D mesh
Jowizo
# Advice on synthetic data generation using 3D mesh

(Asked by: username_0 on 2017-03-27)

I'm developing a deep learning algorithm to find and classify (post consumption) plastic bottles on a conveyor belt. A way to train my model is to get tons of labeled images of each bottle, which for now is not feasible. Another possible option is to make a 3D scan of the bottle and render it in different angles thus creating synthetic training data.

The way I want to obtain the mesh is by using for example ReMake or CapturingReality and 2D images. The problem is that most of the bottles are transparent and are causing problems in those programs. I just ordered Chalk spray to paint them and with a marker add dots allover. Then hopefully I'd be able to render it with the specific material.

My question is, is there a way to add the visual details like the labels and the color of the bottle cap back to the mesh? I'll be using a RGB-D camera so the priorities are getting both the visual cues as well as the depth information as accurate as possible to what the camera will be classifying in a further stage.

Any advise and feedback will be highly appreciated, thanks in advanced!!

**Tags:** rendering, 3d, texture, mesh

> **Comments:**
> **username_1**: Why scan it? I mean most likely there is a model of the bottle somewhere (ask manufacturer) or then modeling it from scratch which usually is not a big deal. Also scanning is more work than taking a hundred pictures.  
> **username_2**: I don't think a rendering approach will work well for you in this case, as a lot of the bottles on your belt will be crushed, which is quite hard to simulate.  
> **username_1**: @username_2 I dont know we did it for testing a reverse vending machine for testing and it seemed to work quite well. Altough it is true that real thing has different levels of data. But it was quite much easier to have digital reolicates of 2000 bottles than actually go and buy them. as quite many of the models were available from the vendor with some emailing and asking nicely.  
> **username_2**: @username_1 Yeah, in general it can be a really effective technique, but if it's for post-consumption plastic bottles you can't expect them to be the same shape as new bottles  
> **username_1**: @username_2 Depends on where you are, in Finland where nearly all bottles and cans are recycled they will in fact all be 90% intact by the time they return as manhandling them means you will not get the deposit back. Glass bottles will be reused as bottles.  
> **username_0**: @username_1 Asking the manufacturer is actually the best idea. You're probably right, i was under the impression that i would need at least a couple of thousand images of each bottle. Can you tell me more about the work you did with the reverse vending machine? How did you guys go about generating the data?  

---

## 0 answers

