<NME> README.md
<BEF> # Introduction

## What it does

**libstreaming** is an API that allows you, with only a few lines of code, to stream the camera and/or microphone of an android powered device using RTP over UDP. 

* Supported encoders includes H.264, H.263, AAC and AMR
* Since version 2.0, a basic support for RTCP has been implemented.
* libstreaming also features a RTSP server for easy remote control of the phone camera and microphone.

The full javadoc documentation of the API is available here: http://dallens.fr/majorkernelpanic/libstreaming/

## How it does it

libstreaming uses various tricks to make streaming possible without the need of any native code. Access to the devices camera(s) and microphone is achieved by simply using a **MediaRecorder**, but configured to write to a **LocalSocket** instead of a regular file ( **MediaStream.java** ). Raw data from the peripheral are then processed in a thread doing syncronous read at the other end of the **LocalSocket**. Voila !

What the thread actually does is this: it waits for data from the peripheral and then packetizes it to make it fit into proper RTP packets that are then sent one by one on the network. The packetization algorithm that must be used for H.264, H.263, AMR and AAC are all specified in their respective RFC:

* RFC 3984 for H.264
* RFC 4629 for H.263
* RFC 3267 for AMR
* RFC 3640 for AAC

Therefore, depending on the nature of the data, the packetizer will be one out of the four packetizers implemented in the rtp package of libstreaming: **H264Packetizer.java**, **H263Packetizer.java**, **AACADTSPacketizer.java** or **AMRNBPacketizer.java**. I took the time to add lots of comments in those files so if you are looking for a basic implementation of one of the RFC mentionned above, they might come in handy.

RTCP packets are also sent by this thread since version 2.0 of libstreaming. Only Sender Reports are implemented.

# Using libstreaming in your app

# Using libstreaming in your app

## Required permissions

```xml
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.CAMERA" />
```

And if you intend to use multicasting, add this one to the list:

```xml
<uses-permission android:name="android.permission.CHANGE_WIFI_MULTICAST_STATE" />
```

## How to stream H.264 and AAC

```java
Session session = SessionBuilder.getInstance()
			.setSurfaceHolder(mSurfaceView.getHolder())
			.setContext(getApplicationContext())
			.setAudioEncoder(SessionBuilder.AUDIO_AAC)
			.setVideoEncoder(SessionBuilder.VIDEO_H264)
			.build();

session.setDestination(destination);
String sdp = session.getSessionDescription();
session.start();
```

The **SessionBuilder** simply facilitates the creation of **Session** objects. The call to **setSurfaceHolder** is needed for video streaming, that should not come up as a surprise since Android requires a valid surface for recording video (It's an ennoying limitation of the **MediaRecorder** API). The call to **setContext** is optional but recommanded, it allows **H264Stream** objects and **AACStream** objects to store and recover data in the **SharedPreferences** of your app. Check the implementation of those two classes to find out exactly what data are stored. 

**Session** objects represents a streaming session to some peer. It contains one or more **Stream** objects that are started (resp. stopped) when the start() (resp. stop()) method is invoked. The method **setDestination** allows you to specify the ip address to which RTP and RTCP packets will be sent. The method **getSessionDescription** will return a SDP of the session in the form of a String.

The complete source code of this example is available here: https://github.com/fyhertz/libstreaming-examples

## How to use the RTSP server

#### Add this to your manifest:

```xml
<service android:name="net.majorkernelpanic.streaming.rtsp.RtspServer"/>
```

If you decide to override **RtspServer** change the line above accordingly.

#### You can change the port used by the RtspServer:

```java
Editor editor = PreferenceManager.getDefaultSharedPreferences(this).edit();
editor.putString(RtspServer.KEY_PORT, String.valueOf(1234));
editor.commit();
```

The port is indeed stored as a String in the preferences, there is a good reason to that. The EditTextPreference object saves its input as a String and cannot easily (one would need to override it) be configured to store it as an Integer.

#### Configure its behavior with the SessionBuilder:

```java
SessionBuilder.getInstance()    
			.setSurfaceHolder(mSurfaceView.getHolder())
			.setContext(getApplicationContext())
			.setAudioEncoder(SessionBuilder.AUDIO_AAC)
			.setVideoEncoder(SessionBuilder.VIDEO_H264);
```

#### Start and stop the server like this:

```java
// Starts the RTSP server
context.startService(new Intent(this,RtspServer.class));
// Stops the RTSP server
context.stopService(new Intent(this,RtspServer.class));
```

# Class diagramm

![Class Diagram](http://dallens.fr/majorkernelpanic/libstreaming/ClassDiagram.png "Class diagram")

*SessionBuilder* and *RtspServer* are the class that you will want to use directly, they make use of everything else in the streaming package.

# Spydroid-ipcamera

Visit [this github page](https://github.com/fyhertz/spydroid-ipcamera) to see how this streaming stack can be used and how it performs.


# Licensing

This streaming stack is available under two licenses, the GPL and a commercial license. *If you are willing to integrate this project into a close source application, please contact me at fyhertz at gmail.com*. Thank you.

<MSG> Updated README

<DFF> @@ -4,26 +4,66 @@
 
 **libstreaming** is an API that allows you, with only a few lines of code, to stream the camera and/or microphone of an android powered device using RTP over UDP. 
 
-* Supported encoders includes H.264, H.263, AAC and AMR
-* Since version 2.0, a basic support for RTCP has been implemented.
-* libstreaming also features a RTSP server for easy remote control of the phone camera and microphone.
+* Android 4.0 or more recent is required.
+* Supported encoders includes H.264, H.263, AAC and AMR.
 
-The full javadoc documentation of the API is available here: http://dallens.fr/majorkernelpanic/libstreaming/
+The first step you will need to achieve to start a streaming session to some peer is called 'signaling'. During this step you will contact the receiver and send a description of the incomming streams. You have three ways to do that with libstreaming.
+
+* With the RTSP client: if you want to stream to a Wowza Media Server, it's the way to go. [The example 3](https://github.com/fyhertz/libstreaming-examples#example-3) illustrates that use case.
+* With the RTSP server: in that case the phone will act as a RTSP server and wait for a RTSP client to request a stream. This use case is illustated in [the example 1](https://github.com/fyhertz/libstreaming-examples#example-1).
+* Or you use libstreaming without using the RTSP protocol at all, and signal the session using SDP over a protocol you like. [The example 2](https://github.com/fyhertz/libstreaming-examples#example-2) illustrates that use case.
+
+The full javadoc documentation of the API is available here: http://majorkernelpanic.com/libstreaming/
 
 ## How it does it
 
-libstreaming uses various tricks to make streaming possible without the need of any native code. Access to the devices camera(s) and microphone is achieved by simply using a **MediaRecorder**, but configured to write to a **LocalSocket** instead of a regular file ( **MediaStream.java** ). Raw data from the peripheral are then processed in a thread doing syncronous read at the other end of the **LocalSocket**. Voila !
+There are three ways on Android to get encoded data from the peripherals:
+
+* With the **MediaRecorder** API and a simple hack
+* With the **MediaCodec** API input buffers, Android 4.1 is required.
+* With the **MediaCodec** API an input surface, only for encoding video and Android 4.3 is required.
+
+### Encoding with the MediaRecorder API
+
+The **MediaRecorder** API was not intended for streaming applications but can be used to retrieve encoded data from the peripherals of the phone.  The trick is to configure a MediaRecorder instance to write to a **LocalSocket** instead of a regular file (see **MediaStream.java**).
+
+This hack has some limitations:
+* Lip sync can be approximative.
+* The MediaRecorder internal buffers can lead to some important jitter. libstreaming tries to compensate that jitter.
+
+It's hard to tell how well this hack is going to work on a phone. It does work well on many devices though.
+
+### Encoding with the MediaCodec API
+
+The **MediaCodec** API do not present the limitations I just mentionned, but has its own issues. There are actually two ways to use the MediaCodec API: with buffers or with a surface.
 
-What the thread actually does is this: it waits for data from the peripheral and then packetizes it to make it fit into proper RTP packets that are then sent one by one on the network. The packetization algorithm that must be used for H.264, H.263, AMR and AAC are all specified in their respective RFC:
+The first method uses calls to [**dequeueInputBuffer**](http://developer.android.com/reference/android/media/MediaCodec.html#dequeueInputBuffer(long)) and [**queueInputBuffer**](http://developer.android.com/reference/android/media/MediaCodec.html#queueInputBuffer(int, int, int, long, int)) to feed the encoder with raw data.
+That seems easy right ? Well it's not, because video encoders that you get access to wit this API are using different color formats and you need to support all of them. A list of those color formats is available [here](http://developer.android.com/reference/android/media/MediaCodecInfo.CodecCapabilities.html). Moreover, many encoders claim support for color formats they don't actually support properly or can present little glitches.
 
-* RFC 3984 for H.264
-* RFC 4629 for H.263
-* RFC 3267 for AMR
-* RFC 3640 for AAC
+All the [**hw**]() package is dedicated to solving those issues. See in particular [**EncoderDebugger**]() class. 
 
-Therefore, depending on the nature of the data, the packetizer will be one out of the four packetizers implemented in the rtp package of libstreaming: **H264Packetizer.java**, **H263Packetizer.java**, **AACADTSPacketizer.java** or **AMRNBPacketizer.java**. I took the time to add lots of comments in those files so if you are looking for a basic implementation of one of the RFC mentionned above, they might come in handy.
+If streaming with that API fails, libstreaming fallbacks on streaming with the **MediaRecorder API**.
 
-RTCP packets are also sent by this thread since version 2.0 of libstreaming. Only Sender Reports are implemented.
+The second solution is to use the [createInputSurface](http://developer.android.com/reference/android/media/MediaCodec.html#createInputSurface()) method. This method is probably the best way to encode raw video from the camera but it requires android 4.3 and up.
+
+The [**gl**]() package is dedicated to using the MediaCodec API with a surface.
+
+It is not yet enabled by default in libstreaming but you can force it with the [**setStreamingMethod**]() method.
+
+### Packetization process
+
+Once raw data from the peripherals has been encoded, it is encapsulated in a proper RTP stream. The packetization algorithm that must be used depends on the format of the data (H.264, H.263, AMR and AAC) and are all specified in their respective RFC:
+
+* RFC 3984 for H.264: **H264Packetizer.java**
+* RFC 4629 for H.263: **H263Packetizer.java**
+* RFC 3267 for AMR: **AMRNBPacketizer.java**
+* RFC 3640 for AAC: **AACADTSPacketizer.java** or **AACLATMPacketizer.java**
+
+If you are looking for a basic implementation of one of the RFC mentionned above, check the sources of corresponding class.
+
+RTCP packets are also sent to the receiver since version 2.0 of libstreaming. Only Sender Reports are implemented. They are actually needed for lip sync.
+
+The [**rtp**]() package handles packetization of encoded data in RTP packets.
 
 # Using libstreaming in your app
 
@@ -36,33 +76,128 @@ RTCP packets are also sent by this thread since version 2.0 of libstreaming. Onl
 <uses-permission android:name="android.permission.CAMERA" />
 ```
 
-And if you intend to use multicasting, add this one to the list:
-
-```xml
-<uses-permission android:name="android.permission.CHANGE_WIFI_MULTICAST_STATE" />
-```
-
 ## How to stream H.264 and AAC
 
+This example is extracted from [this simple android app](https://github.com/fyhertz/libstreaming-examples#example-2). 
+
 ```java
-Session session = SessionBuilder.getInstance()
-			.setSurfaceHolder(mSurfaceView.getHolder())
-			.setContext(getApplicationContext())
-			.setAudioEncoder(SessionBuilder.AUDIO_AAC)
-			.setVideoEncoder(SessionBuilder.VIDEO_H264)
-			.build();
+    protected void onCreate(Bundle savedInstanceState) {
+
+        ...
+
+		mSession = SessionBuilder.getInstance()
+		.setCallback(this)
+		.setSurfaceView(mSurfaceView)
+		.setPreviewOrientation(90)
+		.setContext(getApplicationContext())
+		.setAudioEncoder(SessionBuilder.AUDIO_NONE)
+		.setAudioQuality(new AudioQuality(16000, 32000))
+		.setVideoEncoder(SessionBuilder.VIDEO_H264)
+		.setVideoQuality(new VideoQuality(320,240,20,500000))
+		.build();
+
+		mSurfaceView.getHolder().addCallback(this);
+
+        ...
+
+    }
+
+	public void onPreviewStarted() {
+		Log.d(TAG,"Preview started.");
+	}
+
+	@Override
+	public void onSessionConfigured() {
+		Log.d(TAG,"Preview configured.");
+		// Once the stream is configured, you can get a SDP formated session description
+		// that you can send to the receiver of the stream.
+		// For example, to receive the stream in VLC, store the session description in a .sdp file
+		// and open it with VLC while streming.
+		Log.d(TAG, mSession.getSessionDescription());
+		mSession.start();
+	}
+
+	@Override
+	public void onSessionStarted() {
+		Log.d(TAG,"Streaming session started.");
+        ...
+	}
+
+	@Override
+	public void onSessionStopped() {
+		Log.d(TAG,"Streaming session stopped.");
+        ...
+	}	
+
+	@Override
+	public void onBitrareUpdate(long bitrate) {
+        // Informs you of the bandwidth consumption of the streams
+		Log.d(TAG,"Bitrate: "+bitrate);
+	}
+
+	@Override
+	public void onSessionError(int message, int streamType, Exception e) {
+        // Might happen if the streaming at the requested resolution is not supported
+        // or if the preview surface is not ready...
+        // Check the Session class for a list of the possible errors.
+		Log.e(TAG, "An error occured", e);
+	}
+
+	@Override
+	public void surfaceChanged(SurfaceHolder holder, int format, int width,
+			int height) {
+		
+	}
+
+	@Override
+	public void surfaceCreated(SurfaceHolder holder) {
+        // Starts the preview of the Camera
+		mSession.startPreview();
+	}
+
+	@Override
+	public void surfaceDestroyed(SurfaceHolder holder) {
+        // Stops the streaming session
+		mSession.stop();
+	}
 
-session.setDestination(destination);
-String sdp = session.getSessionDescription();
-session.start();
 ```
 
-The **SessionBuilder** simply facilitates the creation of **Session** objects. The call to **setSurfaceHolder** is needed for video streaming, that should not come up as a surprise since Android requires a valid surface for recording video (It's an ennoying limitation of the **MediaRecorder** API). The call to **setContext** is optional but recommanded, it allows **H264Stream** objects and **AACStream** objects to store and recover data in the **SharedPreferences** of your app. Check the implementation of those two classes to find out exactly what data are stored. 
+The **SessionBuilder** simply facilitates the creation of **Session** objects. The call to **setSurfaceView** is needed for video streaming, that should not come up as a surprise since Android requires a valid surface for recording video (It's an ennoying limitation of the **MediaRecorder** API). On Android 4.3, streaming with no **SurfaceView** is possible but not yet implemented. The call to **setContext** is necessary, it allows **H264Stream** objects and **AACStream** objects to store and recover data using **SharedPreferences**.
+
+**Session** objects represents a streaming session to some peer. It contains one or more **Stream** objects that are started (resp. stopped) when the start() (resp. stop()) method is invoked.
+
+The method **getSessionDescription** will return a SDP of the session in the form of a String. Before calling it, you must make sure that the **Session** has been configured. After calling **configure()** or **startPreview()** on you Session instance, the callback **onSessionConfigured()** will be called.
 
-**Session** objects represents a streaming session to some peer. It contains one or more **Stream** objects that are started (resp. stopped) when the start() (resp. stop()) method is invoked. The method **setDestination** allows you to specify the ip address to which RTP and RTCP packets will be sent. The method **getSessionDescription** will return a SDP of the session in the form of a String.
+**In the example presented above, the Session instance is used in an asyncrounous manner and calls to its methods do not block. You know when stuff is done when callbacks are called.**
+
+**You can also use a Session object in a syncronous manner like that:**
+
+```
+    // Blocks until the all streams are configured
+    try {
+         mSession.syncConfigure();
+    } catch (Exception e) {
+         ...
+    }
+    Strinf sdp = mSession.getSessionDescription();
+    ...
+    // Blocks until streaming actually starts.
+    try {
+         mSession.syncStart();
+    } catch (Exception e) {
+         ...
+    }
+    ...
+    mSession.syncStop();
+```
 
 The complete source code of this example is available here: https://github.com/fyhertz/libstreaming-examples
 
+## How to use the RTSP client
+
+Check out [this page of the wiki](https://github.com/fyhertz/libstreaming/wiki/Using-libstreaming-with-Wowza-Media-Server) and the [example 3](https://github.com/fyhertz/libstreaming-examples#example-3).
+
 ## How to use the RTSP server
 
 #### Add this to your manifest:
@@ -102,12 +237,6 @@ context.startService(new Intent(this,RtspServer.class));
 context.stopService(new Intent(this,RtspServer.class));
 ```
 
-# Class diagramm
-
-![Class Diagram](http://dallens.fr/majorkernelpanic/libstreaming/ClassDiagram.png "Class diagram")
-
-*SessionBuilder* and *RtspServer* are the class that you will want to use directly, they make use of everything else in the streaming package.
-
 # Spydroid-ipcamera
 
 Visit [this github page](https://github.com/fyhertz/spydroid-ipcamera) to see how this streaming stack can be used and how it performs.
@@ -116,4 +245,4 @@ The app. is also available on google play [here](https://play.google.com/store/a
 
 # Licensing
 
-This streaming stack is available under two licenses, the GPL and a commercial license. *If you are willing to integrate this project into a close source application, please contact me at fyhertz at gmail.com*. Thank you.
\ No newline at end of file
+This streaming stack is available under two licenses, the GPL and a commercial license. *If you are willing to integrate this project into a close source application, please contact me at fyhertz at gmail.com*. Thank you.

Updated README