<p>This is more a Computer vision question I think.
I don't know how exactly you can achieve that but some ideas and keywords you may use to find additional resources:</p>

<p>Feature tracking of hand and pen can yield the movement of said things. Assuming you know that the camera is not moving I think that would give you good results. Related is maybe motion tracking. Any information you have about the hand and the camera can in some way be used to make more accurate estimates.</p>

<p>Removing the hand from the video could be much harder. I don't remember that part from my lecture. We copied parts of the image onto the part we want to remove and used image metrics to make sure it fits best possible... But since the hand covers a drawing we don't know much about this could be very hard to do in an convincing way.</p>

CC BY-SA 3.0
2016-01-27T08:42:03.433Z
One could paint on a semitransparent surface and capture image from behind, use wacom tablet trough paper and so on... Reverse difference matte should also give the evolution.
joojaa
CC BY-SA 3.0
2016-01-27T10:14:24.280Z
I understood he had the finished video from somewhere and now wants to reconstruct the process.
Dragonseel
CC BY-SA 3.0
2016-01-27T10:21:52.227Z
Sure but sometimes, one really can not get a good measurement in hindsight. Anyway for the test video tracing works well, and since he has a test video maybe he also hasnt set up the system yet.
joojaa
CC BY-SA 3.0
2016-01-27T00:08:04.783Z
2016-01-27T00:08:04.783Z
Dragonseel
<p>I have a video that record the drawing sequence with a pencil by a painter.
<a href="https://i.sstatic.net/D9UbO.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/D9UbO.png" alt="video screen"></a></p>

<p>I want to reconstruction the sequence from the video.
And I have google some keywords like <em>drawing sequence reconstruction</em>, <em>sketch reconstruction</em>, and have searched like <em>moving object detection</em> but cannot found what I want to find.</p>

<p>I want to know how to detect and recognize the drawing between different frames. I have thought the <em>line detect</em> algorithm may work but fail. And pixel comparison seems to be bad idea. Because that the new drawing may overlap the old ones. </p>

<p>Could you give me some information that I can read or some keywords to search? Sorry about that because I am not familiar with Computer Image Processing.</p>

CC BY-SA 3.0
2016-01-26T16:37:49.393Z
So essentially, you want to remove the hand and pencil from the drawing? So you can have just a time-lapse of the drawing over time? Did I understand you correctly?
RichieSams
CC BY-SA 3.0
2016-01-26T16:40:09.933Z
Actually, I want to know how the painter draw. That is, the sequence of its drawing stroke. The painter will draw by hands on paper. Like that http://imgur.com/FeN1IPy
Jason
CC BY-SA 3.0
2016-01-26T16:40:52.333Z
Remove the hand and pencil may benefit to analyze and it is good also.
Jason
CC BY-SA 3.0
2016-01-26T19:17:59.540Z
So, sort of a vector field drawing showing the path the pencil takes on the paper? For example, each stroke would be an individual curve/arrow.
RichieSams
CC BY-SA 3.0
2016-01-27T07:02:16.060Z
Yes. I just record how painters they paint in daily life actually.
Jason
CC BY-SA 3.0
2016-01-27T08:33:28.030Z
Any demo videos so we can test this? You could simply track the ne dof the pencil. But if it gets occluded then no dice. You could also track the evolution of the shape over frames by difference keying. Or just put the paper on a wacom tablet and change to a wacom ink pen.
joojaa
CC BY-SA 3.0
2016-01-27T09:06:40.550Z
Its my test video, a very simple one. No overlap. http://1drv.ms/1UpF4x3
Jason
CC BY-SA 3.0
2016-01-28T21:42:55.493Z
[Meta discussion](http://meta.computergraphics.stackexchange.com/questions/210/is-reconstruction-of-pen-strokes-from-real-life-video-on-topic) on whether this is on topic.
trichoplax is on Codidact now
<algorithm>
reconstruction drawing sequence from video
Jason
# reconstruction drawing sequence from video

(Asked by: username_0 on 2016-01-26)

I have a video that record the drawing sequence with a pencil by a painter.
[![video screen](https://i.sstatic.net/D9UbO.png)](https://i.sstatic.net/D9UbO.png)

I want to reconstruction the sequence from the video.
And I have google some keywords like *drawing sequence reconstruction*, *sketch reconstruction*, and have searched like *moving object detection* but cannot found what I want to find.

I want to know how to detect and recognize the drawing between different frames. I have thought the *line detect* algorithm may work but fail. And pixel comparison seems to be bad idea. Because that the new drawing may overlap the old ones.

Could you give me some information that I can read or some keywords to search? Sorry about that because I am not familiar with Computer Image Processing.

**Tags:** algorithm

> **Comments:**
> **username_1**: So essentially, you want to remove the hand and pencil from the drawing? So you can have just a time-lapse of the drawing over time? Did I understand you correctly?  
> **username_0**: Actually, I want to know how the painter draw. That is, the sequence of its drawing stroke. The painter will draw by hands on paper. Like that http://imgur.com/FeN1IPy  
> **username_0**: Remove the hand and pencil may benefit to analyze and it is good also.  
> **username_1**: So, sort of a vector field drawing showing the path the pencil takes on the paper? For example, each stroke would be an individual curve/arrow.  
> **username_0**: Yes. I just record how painters they paint in daily life actually.  
> **username_2**: Any demo videos so we can test this? You could simply track the ne dof the pencil. But if it gets occluded then no dice. You could also track the evolution of the shape over frames by difference keying. Or just put the paper on a wacom tablet and change to a wacom ink pen.  
> **username_0**: Its my test video, a very simple one. No overlap. http://1drv.ms/1UpF4x3  
> **username_3**: [Meta discussion](http://meta.computergraphics.stackexchange.com/questions/210/is-reconstruction-of-pen-strokes-from-real-life-video-on-topic) on whether this is on topic.  

---

## 1 answers

### Answer by username_4 on 2016-01-27. Score: 2 | Accepted answer

This is more a Computer vision question I think.
I don't know how exactly you can achieve that but some ideas and keywords you may use to find additional resources:

Feature tracking of hand and pen can yield the movement of said things. Assuming you know that the camera is not moving I think that would give you good results. Related is maybe motion tracking. Any information you have about the hand and the camera can in some way be used to make more accurate estimates.

Removing the hand from the video could be much harder. I don't remember that part from my lecture. We copied parts of the image onto the part we want to remove and used image metrics to make sure it fits best possible... But since the hand covers a drawing we don't know much about this could be very hard to do in an convincing way.

> **Comments:**
> **username_2**: One could paint on a semitransparent surface and capture image from behind, use wacom tablet trough paper and so on... Reverse difference matte should also give the evolution.  
> **username_4**: I understood he had the finished video from somewhere and now wants to reconstruct the process.  
> **username_2**: Sure but sometimes, one really can not get a good measurement in hindsight. Anyway for the test video tracing works well, and since he has a test video maybe he also hasnt set up the system yet.  

---

