Elphel Camera Remote Audio using GStreamer
From ElphelWiki
Revision as of 15:34, 10 June 2007 by Luxigo (talk | contribs)
You can browse a better looking version of this how-to at: http://luxigo.tiddlyspot.com/#%5B%5BElphel%20Camera%20Remote%20Audio%20HOW-TO%5D%5D
---------------------------------------------------------------------------- HOW TO ADD AUDIO TO YOUR ELPHEL CAMERA VIDEO STREAM USING GSTREAMER: ---------------------------------------------------------------------------- In two words and three lines we have to: - demultiplex (demux) the incoming ogg stream to read the mjpeg stream, - grab the sound from an audio source, - multiplex (mux) the mjpeg and the audio streams into an ogg container. Paste the script below in a new file or download the full version here, change the settings, save it as "soundmux.sh", and make it executable with: chmod +x soundmux.sh then run it with: ElphelOgm | soundmux.sh Simplified soundmux.sh: (download the full version here: http://luxigo.ath.cx/elphel/gstreamer/soundmux.sh) #!/bin/sh # audio settings AUDIO_DEVICE="hw:0,0" AUDIO_BITRATE=48000 # output file name OUTFILE="out.ogg" DEBUG="-v -m" gst-launch-0.10 $DEBUG \ \ oggmux name=mux \ ! filesink location="$OUTFILE" \ \ fdsrc fd=0 \ ! queue \ ! oggparse \ ! oggdemux \ ! ogmvideoparse \ ! mux. \ \ alsasrc \ device="$AUDIO_DEVICE" \ ! queue \ ! audioconvert \ ! vorbisenc bitrate=$AUDIO_BITRATE \ ! mux. # Description of the gst-launch command above: # - the muxed stream (the output from the oggmux element) will be # stored in a file (filesink) # - The video comes from ElphelOgm through stdin (fdsrc fd=0). # - The ogg stream is parsed for security (optional?) and demuxed. # - The ogmvideo stream is also parsed for security (optional?). # - The resulting mjpeg stream is sent to the ogg muxer through the # "mux." element (defined with "oggmux name=mux"). # (This syntax could also be used for oggdemux but here there's only # a single stream in the incoming ogg container so we dont need to # complicate things here) # - The sound comes from an alsa device through alsasrc, # - The sound is converted in a suitable format then encoded in vorbis, # before being muxed with the video stream. ---------------------------------------------------------------------------- BROADCAST THE VIDEO MULTIPLEXED WITH AUDIO USING ICECAST 2 ---------------------------------------------------------------------------- In the same time you add audio, you can resize and transcode the video stream in theora in order to broadcast the resulting stream on your icecast server (if your machine can handle the frame size at the given framerate). Paste the script below in a new file or download the full version here, change the default settings if required, then save it as "oggmicecast.sh" and make it executable with: chmod +x oggmicecast.sh Then run it with: ElphelOgm | oggmicecast.sh Finally you can use mplayer to watch the stream: gmplayer http://icecast:8000/elphel.ogg.m3u Simplified oggmicecast.sh: (download the full version here:http://luxigo.ath.cx/elphel/gstreamer/oggmicecast.sh) #!/bin/sh # video settings VIDEO_QUALITY=51 # between 0 and 63 (51 is ~80%) WIDTH=480 # 4/3: ... 512x384, 480x360, 353x264, ... HEIGHT=360 # 16/9: ... 640x360, 512x288, 384x216, ... # audio settings AUDIO_BITRATE=48000 # ..., 32000, 48000, 64000, ... AUDIO_DEVICE="hw:0,0" # ALSA notation # icecast settings ICECAST_SERVER=icecast ICECAST_SERVER_PORT=8000 ICECAST_SERVER_PW="hackme" ICECAST_STREAMNAME="Elphel Camera" ICECAST_MOUNTPOINT="/elphel.ogg" DEBUG="-v -m" # starting gstreamer gst-launch-0.10 $DEBUG \ \ oggmux name=mux \ ! queue \ ! shout2send \ ip=$ICECAST_SERVER \ port=$ICECAST_SERVER_PORT \ password="$ICECAST_SERVER_PW" \ streamname="$ICECAST_STREAMNAME" \ protocol=3 \ sync=true \ mount="$ICECAST_MOUNTPOINT" \ \ fdsrc fd=0 \ ! queue \ ! oggdemux \ ! ogmvideoparse \ ! jpegdec \ ! queue \ ! videoscale \ method=1 ! video/x-raw-yuv,width=$WIDTH height=$HEIGHT \ ! theoraenc quality=$VIDEO_QUALITY \ ! mux. \ \ alsasrc \ device="$AUDIO_DEVICE" \ ! audioconvert \ ! vorbisenc bitrate=$AUDIO_BITRATE \ ! mux. ---------------------------------------------------------------------------- REMOTE MICROPHONE: ---------------------------------------------------------------------------- The microphone can be located on another machine, far from the camera. Replace the "alsasrc" paragraph in oggmicecast.sh on the server with: tcpserversrc \ host=local_interface_ip \ port=9003 \ protocol=1 \ ! mux. and run the script below (micsend.sh) on the client machine where you plugged the microphone, giving the ip you set for the tcpserversrc host on the other side as last argument: micsend.sh <hostname> Simplified micsend.sh: (download the full version here:http://luxigo.ath.cx/elphel/gstreamer/micsend.sh) # audio settings AUDIO_DEVICE="hw:0,0" AUDIO_BITRATE=48000 # network settings TCP_SERVER_PORT=9003 TCP_SERVER=host_running_oggmicecast DEBUG="-v -m" gst-launch-0.10 $DEBUG \ \ alsasrc \ device=$AUDIO_DEVICE \ ! audioconvert \ ! queue \ ! vorbisenc bitrate=$AUDIO_BITRATE \ ! tcpclientsink \ host=$TCP_SERVER \ port=$TCP_SERVER_PORT \ max-lateness=150000000 \ sync=true \ protocol=1 ---------------------------------------------------------------------------- GSTREAMER HINTS: ---------------------------------------------------------------------------- - Use "!" pipe the output from the previous gstreamer element in the next one. - Try "gst-inspect-0.10" to list available gst-launch elements (commands) and "gst-inspect-0.10 <command>" to display syntax details for a single command. - An empty line in the formatted gst-launch command line layout used in this page denotes a distinct pipe. ---------------------------------------------------------------------------- KNOWN PROBLEMS: ---------------------------------------------------------------------------- Trying to use a different thread to recompress video (using gst-launch brace syntax), we were not able to synchronize audio with video. Using ElphelOgm-0.9.7 in unicast mode and gstreamer-0.10, we experienced systematic crash in oggdemux (but i cant say for sure it never worked in the past or if it is an ElphelOgm encapsulation problem or a gstreamer oggdemux problem). So actually we are using ElphelOgm 0.9.9 in multicast mode, but.. Using ElphelOgm in multicast mode, icecast server 2.3.1, and finally vlc 0.8.6 to watch the stream from the icecast server, we experienced image freeze in vlc after some time of playback, with all subsequent frames dropped. It was not the case using ElphelOgm-0.9.7 in unicast mode. (maybe vlc had problems decoding theora because the network was overloaded by many other streams and some packets were lost) However MPlayer1.0rc1-0ubuntu9 did play the same stream through the same overloaded switch without any problem ;) Maybe you must recompile your icecast server so that it supports theora: Try "ldd /usr/bin/icecast" (/usr/bin/icecast2 in ubuntu) to see if yours is compiled with theora support enabled. ---------------------------------------------------------------------------- LINKS: ---------------------------------------------------------------------------- http://gstreamer.freedesktop.org/ (see GSTREAMER HINTS) http://www.icecast.org/ http://www.mplayerhq.hu/ http://www.videolan.org/ (see KNOWN PROBLEMS)