<p>It is possible to do what you describe, but I'm afraid it is not a trivial process. Actually, this will be very tied to the Operating Systems you are targeting and possibly also requiring specific tweaks for the given game/application.</p>

<p>I would start looking into <a href="https://en.wikipedia.org/wiki/DLL_injection">DLL injection</a> techniques, which should allow, for instance, intercepting calls to the Direct3D (Windows) or OpenGL APIs, which you can then use to copy de framebuffers of the application. A quick Internet search turns up <a href="http://blog.opensecurityresearch.com/2013/01/windows-dll-injection-basics.html">this</a> and <a href="http://www.evilsocket.net/2015/05/01/dynamically-inject-a-shared-library-into-a-running-process-on-androidarm/">this</a> with detailed explanation. </p>

<p>I once wrote a small-scale OpenGL interceptor on MacOS by injecting a custom shared module, but it was under very controlled circumstances and making a lot of assumptions, such as having root privileges in the system, so for a production tool, it would be a lot more complicated. You can find another very interesting project <a href="https://graphics.stanford.edu/~mdfisher/D3D9Interceptor.html">here</a> about intercepting the D3D API to install an automated AI bot on Starcraft, which should touch on very similar concepts to what you are looking for.</p>

<p>Once you manage to intercept the underlying rendering API, you could then perhaps hook your code into each <code>SwapBuffers</code> or equivalent call to just copy the previous framebuffer before the swap, then save it (this is where something like <code>glReadPixels</code> would come into play).</p>

<p>Copying and saving the framebuffer in an efficient manner is also challenging. The easy way is to just dump each frame as an uncompressed RGB image, but then you'll wind up with hundreds of gigabytes of data for just a couple of minutes of gameplay, so unless you have a nice HDD array sitting at the corner of your table <code>;)</code>, you'll need to look into compressing the frames somehow. </p>

<p>The downside of compressing the frames now is that it takes a lot of processing, so a naive approach might turn the once interactive application you are trying to record into an interactive slide-show <code>:(</code>. So at this stage you'd have to start thinking about optimizations.</p>

<p>Don't know of any projects out there providing a library for efficient frame capture for games, but this would certainly be something nice to have! I think one thing that might be holding such project back is, like I mentioned at the beginning, that this is a very system dependent thing, so cases of use will most likely be limited to a single platform or OS.</p>

CC BY-SA 3.0
2015-10-25T10:10:03.303Z
Thanks for the answer. I think that ultimately, I will instead of saving images, stream them to a remote machine, so copying the framebuffer may be the way to go. You seem to have a fair bit of experience with this stuff. Are you able to expand on any of the questions that I posed? Thanks for the help!
pookie
CC BY-SA 3.0
2015-10-25T17:13:52.243Z
@pookie Hey! Sure, I can expand this. Is there anything specifically you'd like me to comment on? For your questions, I think 1&2 can be done with the DLL hooks, so be sure to read the links I've included (specially the one about starcraft). 3&4 will be API specific, but on OpenGL, you can start with glReadPixels and work from there if you find it to be too slow. My experience is from recording frames from my own games plus some DLL hacking done in the past, but I've never combined both in the attempt of recording frames from a third party app ;)
glampert
CC BY-SA 3.0
2015-10-25T17:23:39.877Z
Thank you! I've checked the links - windows DLL injection and starcraft link (pretty awesome) seem to be good starting points. I will take a deapre look into glReadPixels, too. In your opinion, which did you find easier to work with and which did you find faster: DirectX or OpenGL?
pookie
CC BY-SA 3.0
2015-10-25T18:08:27.037Z
@pookie, if you have some programming experience, I think Direct3D will be much easier to use, the libraries are better designed. OpenGL has too may "rough edges" in my opinion (this coming from someone that uses OpenGL more than D3D). D3D is Windows only, so that might be a negative for you, don't know. Performance-wise, I don't think there's going to make much of a difference which one you use. The best advice I can probably give you is: first figure out how to get it working, then worry about optimizations.
glampert
CC BY-SA 3.0
2015-10-25T18:11:45.210Z
Thanks, I've a fair bit of c# knowledge - not sure how I will take to D3D, but I will give it a stab.
pookie
CC BY-SA 3.0
2015-10-25T18:30:15.723Z
Ah, ok, if you're familiar with OOP programming, then D3D will definitely be easier. OpenGL is all procedural and uses a bunch of global state, that's mostly what I meant about rough edges. Best of luck to you!
glampert
CC BY-SA 3.0
2015-10-24T17:49:58.717Z
2015-10-24T17:57:52.740Z
2015-10-24T17:57:52.740Z
glampert
<p>I'd like to "hook" into a running game, say Mario Bros, and capture each frame rendered... saving that frame to an image file. A good example of something similar is FRAPS. --Note: I don't want to capture the whole screen/desktop. I want to capture a targeted window.</p>

<p>I have had a look at <a href="https://obsproject.com/" rel="noreferrer">OBS</a> (Open Broadcasting Software) but it is not particularly fast. Don't get me wrong, it's great software, but unfortunately there is no/poor documentation, making a massive project written in c and c++ nearly inaccessible for a new-to-c++ programmer.</p>

<p>I've also had a look at <a href="http://gaminganywhere.org/perf.html" rel="noreferrer">GamingAnywhere</a>, but unfortunately, I am unable to get it to work, there is very little/no documentation, runs in VS2010 only and is messy (with poor variable naming). However, it is a research project and so it is understandably undocumented and messy.</p>

<p>I know that this can be done with OpenGL, GDI and with Direct3D, but I am unable to find some good examples on the net. </p>

<p>I read that glReadlPixels (using OpenGL) can be used and I've read the <a href="https://www.opengl.org/sdk/docs/man2/xhtml/glReadPixels.xml" rel="noreferrer">documentation</a>, but the post mentioned nothing about hooking into a running game/application graphics.</p>

<p><strong>Questions:</strong></p>

<ol>
<li><p>Can I hook into the graphics of a game developed with OpenGL, using,
say, Direct3D? Does the library used for hooking have to be the same
as the one used by the game?</p></li>
<li><p>How do I hook into the game's rendered frames so that I can output
those frames to image files or to a video file? (Just some links or
brief explanation of what I need to do would be great)</p></li>
<li><p>BackBuffer - I read that it is very fast accessing the BackBuffer to
retrieve the frames. Does someone have an example for me on how to
do this with the latest libraries? I have found that most examples
are out of date.</p></li>
<li><p>For my purposes, is there any clearly "this is faster than that"? What I mean is, would, say, OpenGL, be faster for my purposes? </p></li>
</ol>

<p>If someone knows of an open-source project (that does essentially what I need) that is actively developed and well documented, I'd love to know about it.</p>

CC BY-SA 3.0
2015-10-24T22:45:32.967Z
Now this appears to be Mac only, which may not work for you with Direct3D, but [Syphon Inject](http://syphon.v002.info) which OBS uses on Mac can capture game windows that use OpenGL _much_ faster than standard video capture.  Syphon "allows applications to share frames - full frame rate video or stills - with one another in realtime" to quote the website.
Gliderman
<opengl><direct3d>
Screen capture of game video
pookie
# Screen capture of game video

(Asked by: username_0 on 2015-10-24)

I'd like to "hook" into a running game, say Mario Bros, and capture each frame rendered... saving that frame to an image file. A good example of something similar is FRAPS. --Note: I don't want to capture the whole screen/desktop. I want to capture a targeted window.

I have had a look at [OBS](https://obsproject.com/) (Open Broadcasting Software) but it is not particularly fast. Don't get me wrong, it's great software, but unfortunately there is no/poor documentation, making a massive project written in c and c++ nearly inaccessible for a new-to-c++ programmer.

I've also had a look at [GamingAnywhere](http://gaminganywhere.org/perf.html), but unfortunately, I am unable to get it to work, there is very little/no documentation, runs in VS2010 only and is messy (with poor variable naming). However, it is a research project and so it is understandably undocumented and messy.

I know that this can be done with OpenGL, GDI and with Direct3D, but I am unable to find some good examples on the net.

I read that glReadlPixels (using OpenGL) can be used and I've read the [documentation](https://www.opengl.org/sdk/docs/man2/xhtml/glReadPixels.xml), but the post mentioned nothing about hooking into a running game/application graphics.

**Questions:**

1. Can I hook into the graphics of a game developed with OpenGL, using,
   say, Direct3D? Does the library used for hooking have to be the same
   as the one used by the game?
2. How do I hook into the game's rendered frames so that I can output
   those frames to image files or to a video file? (Just some links or
   brief explanation of what I need to do would be great)
3. BackBuffer - I read that it is very fast accessing the BackBuffer to
   retrieve the frames. Does someone have an example for me on how to
   do this with the latest libraries? I have found that most examples
   are out of date.
4. For my purposes, is there any clearly "this is faster than that"? What I mean is, would, say, OpenGL, be faster for my purposes?

If someone knows of an open-source project (that does essentially what I need) that is actively developed and well documented, I'd love to know about it.

**Tags:** opengl, direct3d

> **Comments:**
> **username_1**: Now this appears to be Mac only, which may not work for you with Direct3D, but [Syphon Inject](http://syphon.v002.info) which OBS uses on Mac can capture game windows that use OpenGL _much_ faster than standard video capture.  Syphon "allows applications to share frames - full frame rate video or stills - with one another in realtime" to quote the website.  

---

## 1 answers

### Answer by username_2 on 2015-10-24. Score: 9 | Accepted answer

It is possible to do what you describe, but I'm afraid it is not a trivial process. Actually, this will be very tied to the Operating Systems you are targeting and possibly also requiring specific tweaks for the given game/application.

I would start looking into [DLL injection](https://en.wikipedia.org/wiki/DLL_injection) techniques, which should allow, for instance, intercepting calls to the Direct3D (Windows) or OpenGL APIs, which you can then use to copy de framebuffers of the application. A quick Internet search turns up [this](http://blog.opensecurityresearch.com/2013/01/windows-dll-injection-basics.html) and [this](http://www.evilsocket.net/2015/05/01/dynamically-inject-a-shared-library-into-a-running-process-on-androidarm/) with detailed explanation.

I once wrote a small-scale OpenGL interceptor on MacOS by injecting a custom shared module, but it was under very controlled circumstances and making a lot of assumptions, such as having root privileges in the system, so for a production tool, it would be a lot more complicated. You can find another very interesting project [here](https://graphics.stanford.edu/~mdfisher/D3D9Interceptor.html) about intercepting the D3D API to install an automated AI bot on Starcraft, which should touch on very similar concepts to what you are looking for.

Once you manage to intercept the underlying rendering API, you could then perhaps hook your code into each `SwapBuffers` or equivalent call to just copy the previous framebuffer before the swap, then save it (this is where something like `glReadPixels` would come into play).

Copying and saving the framebuffer in an efficient manner is also challenging. The easy way is to just dump each frame as an uncompressed RGB image, but then you'll wind up with hundreds of gigabytes of data for just a couple of minutes of gameplay, so unless you have a nice HDD array sitting at the corner of your table `;)`, you'll need to look into compressing the frames somehow.

The downside of compressing the frames now is that it takes a lot of processing, so a naive approach might turn the once interactive application you are trying to record into an interactive slide-show `:(`. So at this stage you'd have to start thinking about optimizations.

Don't know of any projects out there providing a library for efficient frame capture for games, but this would certainly be something nice to have! I think one thing that might be holding such project back is, like I mentioned at the beginning, that this is a very system dependent thing, so cases of use will most likely be limited to a single platform or OS.

> **Comments:**
> **username_0**: Thanks for the answer. I think that ultimately, I will instead of saving images, stream them to a remote machine, so copying the framebuffer may be the way to go. You seem to have a fair bit of experience with this stuff. Are you able to expand on any of the questions that I posed? Thanks for the help!  
> **username_2**: @username_0 Hey! Sure, I can expand this. Is there anything specifically you'd like me to comment on? For your questions, I think 1&2 can be done with the DLL hooks, so be sure to read the links I've included (specially the one about starcraft). 3&4 will be API specific, but on OpenGL, you can start with glReadPixels and work from there if you find it to be too slow. My experience is from recording frames from my own games plus some DLL hacking done in the past, but I've never combined both in the attempt of recording frames from a third party app ;)  
> **username_0**: Thank you! I've checked the links - windows DLL injection and starcraft link (pretty awesome) seem to be good starting points. I will take a deapre look into glReadPixels, too. In your opinion, which did you find easier to work with and which did you find faster: DirectX or OpenGL?  
> **username_2**: @username_0, if you have some programming experience, I think Direct3D will be much easier to use, the libraries are better designed. OpenGL has too may "rough edges" in my opinion (this coming from someone that uses OpenGL more than D3D). D3D is Windows only, so that might be a negative for you, don't know. Performance-wise, I don't think there's going to make much of a difference which one you use. The best advice I can probably give you is: first figure out how to get it working, then worry about optimizations.  
> **username_0**: Thanks, I've a fair bit of c# knowledge - not sure how I will take to D3D, but I will give it a stab.  
> **username_2**: Ah, ok, if you're familiar with OOP programming, then D3D will definitely be easier. OpenGL is all procedural and uses a bunch of global state, that's mostly what I meant about rough edges. Best of luck to you!  

---

