Search Options | Help | Site Map | Cultivate Web Site
		Search Disabled

Streaming Video: A Look Behind the Scenes

By Jim Strom - May 2001

Over the last few years there has been a dramatic improvement in the quality of IP-based network media technologies. Both real time and on-demand media can now be created, served and played at the desktop using PC-based platforms and software freely available across the Internet.

In our second article on this topic Jim Strom, responsible for the Future is Hybrid Multimedia Presentations [1], gives us a behind the scenes look at what can be achieved with streaming video using a number of examples and case studies.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Introduction

If you want to see a good example of streaming video then have a look at the TERENA (Trans European Research Network Association) conference given in Lund, Sweden in June 1999 [2]. The whole conference was streamed out live across the Internet. When I watched it, I realised that streaming technology for distance learning had finally arrived. The quality of the video and audio was much better than anything I had ever seen or heard before. It was as good as actually being there at the conference. In fact it was better, because during parallel presentations I could pull down the separate streams and easily switch between sessions without having to shuffle between rooms. That’s the real benefit of interactive Web media – doing things that can’t be done in real life.

Behind the scenes there was a lot of technical preparation and resources needed to achieve the quality of the video stream in Lund, however the techniques can all be replicated on PCs using freely available software. The stream from Lund shows the presenter’s slides running alongside the video. This was done using SMIL (Synchronised Multimedia Integration Language), which defines the layout and synchronisation of the different media clips in the stream. When we set up the Advanced Telematics Centre (ATC) [3] in 1999, at the University of Manchester, we recognised the importance of this technique for e-learning and have since used it extensively in both live and on-demand developments with streaming media. This paper illustrates 2 case study examples of streaming productions that the ATC has done using SMIL.

Some Basic Concepts

In order to play smoothly, video data needs to be available continuously and in the proper sequence without interruption. Until fairly recently, it had to be downloaded in its entirety to the PC before it could be played. With streaming, the file remains on the server. The initial part is copied to a buffer on the PC and then, after a short delay, called ‘preroll’, starts to play and continues as the rest of the file is being pulled down. Streaming provides a steady method of delivery controlled by interaction between the PC and the server. The server regulates the stream according to network congestion and thereby optimises the presentation on the PC.

There are 3 software components involved in streaming:

• The Player: The client software that must be loaded on the PC in order to play back the stream. This may be run as a stand-alone application or launched through a plug-in for the browser.
• The Video Server: The software that handles the distribution of streams. The video server is I/O intensive so the machine to support it needs to have fast disc access and plenty of free storage capacity to hold the content.
• The Encoder: The software that compresses an audio/video source into a file that can be streamed over the network. Encoding is CPU-intensive so the machine that supports it needs to have a fast processor and sufficient memory.

Content can be On-demand or Broadcast

On-demand content is controlled by the client. The user can select a pre-recorded stream and also freely choose when to view it. Furthermore the user can control the video stream - pausing, jumping ahead/back, restarting, etc – just as with a video recorder. On the other hand, broadcast content is controlled and scheduled by the server. The content is only made available for viewing at selected times. The viewer can only watch the stream as it is being transmitted without any control over it, just as with a television or radio broadcast. Broadcast content can be scheduled to come from an archived file or can be a live transmission from an external audio/video device such as a camera or video recorder.

Streams can be Unicast or Multicast

A unicast stream refers to a single link from the video server down to the client for access to either on-demand or broadcast content. A single video server is able to handle many simultaneous unicast links to clients accessing the same or different content. Unicast streams can range from 20Kbps to more than 1Mbps so an important consideration here is the aggregate bandwidth needed by the video server. 60 unicasts, for example, coming down together at 20Kbps each, are going to amount to a total bandwidth of 1.2Mbps (60x20Kbps) over the video server’s local network.

Multicast gets around this problem by sending out a single stream that can be picked up by any number of clients, thus saving network bandwidth. In the case of the previous example, that would mean 20Kbps bandwidth usage for 1, 60 or 6000 simultaneous viewers watching the same broadcast. The number of viewers is immaterial. However, multicast only offers the client the opportunity to join a live or scheduled broadcast. The user has no control over the stream and cannot stop or restart the transmission. Multicasting is controlled by the network infrastructure itself and is dependent on the routers being multicast-capable. Institutions may be able to offer this internally over their own intranets if their routers are enabled. Also, over the academic network, we have access to Mbone – the academic multicast backbone. But outside of this, across commercial Internet, there is very little opportunity to use multicast at the moment. Unicast is therefore the predominant form of streaming for global access.

Web Server vs Video Server

On-demand content can be delivered from a Web server rather than having to set up and manage a separate video server. When a user requests the video file, it starts to be copied down to the PC using HTTP like any other Web data. Control is passed to the player and the stream plays as the rest of the file is being brought down. However HTTP and its underlying TCP/IP protocol are designed simply to transfer text and images reliably to the client and offers no inherent control over the stream. This leads to a bursty form of transfer rather than a steady delivery. In practice it is better to use a video server to regulate the transfer and give a smooth playback at the client end without rebuffering interruptions. For a useful overview of the pros and cons of this, see Microsoft’s white paper on Streaming Methods [4].

Implementing Streaming Video

The two main contenders in the streaming arena are RealNetworks’ RealSystem G2 and Microsoft’s Windows Media Technologies. Both suppliers provide the basic three streaming software components for free, downloadable across the Internet. Apple’s QuickTime is also now capable of streaming, with the release of QuickTime 5. Table 1 shows the platforms and provision of the player, server and encoder software.

Table 1: Streaming Media Software from Real Networks, Microsoft and Apple.

Both RealSystem G2 and Windows Media Technologies embody a rich set of streaming management features including multicast capability, security and authentication resources and tools for streaming slide presentations. Both video servers provide Dynamic Stream Control so that the stream rate is adjusted according to network conditions. Also both products offer Multiple Bit Rate Encoding. This is the ability to encode a single file that can be streamed out to clients at different data rates according to their access bandwidth. The server will automatically select the appropriate encoding suited to the client’s bandwidth when the connection is made.

In an ideal world you might expect to view any content on any player. However, in general, content generated by one supplier’s encoder is only viewable through that supplier’s player. Also watch out for version incompatibility between players and content. For example, content encoded with the latest version of a supplier’s encoder may turn out to be not viewable by earlier versions of the player. This problem occurs with versions of RealPlayer.

Streaming Protocol and File Format

The video server uses a streaming protocol to manage the flow of data and control information with the client. RealSystem uses RTSP (Real Time Streaming Protocol), a standards-based client-server protocol for streaming media. Windows Media uses a proprietary protocol called Microsoft Media Server (MMS). Each system has its own file format for the streamed data. RealSystem encodes into a .rm file format. Windows Media encodes into an .asf or .wmv file format. You can see examples of these with the following references: [5], [6], [7], [8], [9], [10] representing both RealSystem and Windows Media encodings for 56Kbps, ISDN and DSL access rates. [You will need to have RealPlayer 8 and the Windows Media 7 Player installed to view these.] In each case, the same content has been encoded so a direct comparison of video and audio quality can be made. The playback window size is 320x240 pixels.

The content used in these examples is a 1-minute clip of Professor Frank Sumner from the University of Manchester (Figure 1). The clip was taken from a videoconference link with an assembled group in Antwerp where Professor Sumner was talking about his experiences at the time of the Manchester Baby (the first stored program computer that ran its first program on 21 June 1948).

Figure 1: Antwerp video clip used to compare the quality of Real Video and Windows Media

Buffering

The audio and video will come down as separate streams. During the preroll, the player will buffer a small amount before starting to play with the aim of bringing down the remainder as the clip is playing. If the client’s bandwidth is not sufficient to sustain the stream then there will be frequent pauses as rebuffering takes place. You can see this happening in the ‘Manchester Baby’ examples if the ISDN or DSL clip is pulled down over a modem link. The server will always try to maintain the audio stream at the expense of the video stream since interruptions in the audio are more noticeable than changes in the video.

Capacity Planning

Media files can become very large and therefore take up considerable disc space. In producing the ‘Manchester Baby’ clip, the original video was first captured for editing in an uncompressed, AVI (Audio Video Interleave) format. This 1-minute clip at 24-bit colour depth, at 15 fps and with a 320x240 pixel frame size took up 267Mb! When compressed however for the two encoding systems, it came down to around 350Kb (56K), 800Kb (ISDN) and 4,300Kb (DSL). Projecting from these figures, a 1-hour stream for access rates of up to 250Kbps is going to take up around 260Mb of storage on the video server.

SMIL you’re on Video

SMIL (Synchronised Multimedia Integration Language) is the XML complaint markup language that provides a time-based synchronised environment to stream audio, video, text, images and animation. SMIL is the officially recognised standard of the World Wide Web Consortium. RealSystem G2 and QuickTime 5 are SMIL compliant. The current Windows Media 7 player is not, however the next release of the Windows Media, due out later this year, will be compliant, albeit with a Microsoft flavoring. So SMIL is set to become the common media mark-up language across all the three steaming platforms.

A SMIL file (with a .smi extension) defines the layout and sequencing of the media clips. Sequential (<seq>) and parallel (<par>) tags allow you to specify that clips should be played either one after another or at the same time.

Using a SMIL file to control a presentation means that a slide stream can be run in parallel with a presenter’s video stream, scrolling or ticker text can appear and live links to other Web pages or media can be built in. RealNetworks have added some extensions of their own to the SMIL standard and have defined two additional data types:

• RealPix (with a .rp extension) allowing you to stream a sequence of graphic images as a series of jpegs and to control the timing;
• RealText (with a .rt extension) allowing you to provide a stream of scrolling or tickertape text.

Some Examples of Production

The following two case studies, together with some general guidance on production, provide examples of using SMIL with RealSystem G2.

Case Study 1

This event was a student fashion show held by UMIST Department of Textiles. The requirement was to stream it out live over the Web and to provide an archived copy on the server. The encoder used a Windows98 PC (Celeron 400, 64Mb) together with a Hauppauge WinTV capture card and the free RealProducer Basic software. The encoded stream was fed over a LAN to a video server, (WinNT4, dual PII 400, 128Mb + SCSI drive), running the free RealServer Basic software. The event was advertised on the Department’s Web site with a link to a SMIL file on the video server. The SMIL file produced a presentation (Figure 2) with 3 media inserts: the UMIST logo, some ticker text produced using RealText and the live video window. The stream was archived onto the encoder PC as it was broadcast. The source video was also captured onto digital videotape and used to generate an edited version of the Webcast to go up on the server. The resulting stream can be seen at [11].

Figure 2: UMIST fashion show live Webcast

The SMIL code used for this is shown in Table 2. Basically the code defines the layout for the different areas in the presentation window and then, in the ‘body’, specifies the source and relationship of the three streams. The surrounding ‘par’ tags cause the streams to be played back in parallel.

A further example of a live Webcast done by the ATC can be seen at [12]. This was an outside event for the Mottram Millennium festival (Cheshire) where a civil war re-enactment was captured in the field (literally) and relayed back over a wireless link to a nearby cybercafe for encoding. From there it was exported over ISDN to the video server in Manchester. The example shows the use of scrolling text to support the video presentation.

Table 2: SMIL code used for the UMIST student fashion show.

Some things to bear in mind with live streaming

• Always fully test the encoder equipment and network links beforehand;
• Disable any screensaver on the encoder and terminate any unnecessary applications;
• Try to use a more reliable Windows NT or 2000 platform rather than Windows 98;
• Make sure that clients have the right player for the stream. For example, if you plan to encode using RealProducer 8 then your viewers will need to download RealPlayer 8 beforehand;
• Put out a test video beforehand so that viewers can establish that their client system is working correctly;
• Don’t simply rely on capturing the stream as an archive on the encoder machine. Capture it onto videotape at the same time. Editing for on-demand will be easier from videotape;
• Check who the Webcast is intended for. If it is many for modem users, then you might encode for 28.8/56Kbps with a small window size, eg 200x150, and at 12.5 fps. If it is for LAN/broadband viewing, then you might encode for DSL/LAN with a larger window, eg 320x240 and at 25 fps. Don’t try to do it full screen unless you have a hardware encoding system;
• Use a Web-based administrator tool (included with the video server) to watch what is happening on the server and to monitor how many streams are being pulled down.

Case Study 2

This event was a public seminar on xDSL broadband access technologies with speakers from the industry. The requirement was to obtain a good quality video of the presentations and then to create the streaming media afterwards. The set up was to use a camera crew with 2 cameras, lights, fixed and roving mics, audio and video mixing, etc.

The videotapes were edited and then used to generate the streaming video using a NT workstation running RealProducer Basic. The resulting presentation is shown in Figure 3. The speaker’s slides were converted to jpegs and encoded using RealPix to be synchronised with the presenter’s video.

Figure 3: xDSL Presentation by Patrick De Boeck, Telindus K-Net

The SMIL file used for this is shown in Table 3. There are 6 media inserts covering the presenter’s video, slides, Web site (via logo) and email links for the speaker and the ATC. The body of the code contains a switch element to differentiate 3 different streaming bands - 150Kbps, 80Kbps, 20Kbps - and to stream a different encoding of the presentation slides and the presenter’s video according to the access bandwidth at the client end. The resulting presentation can be seen at [13].

Table 3: SMIL code used for the Telindus K-Net xDSL presentation.

Some things to bear in mind with on-demand encoding

• Use a switch element in the SMIL code to select different presentation encodings rather than use a single multiple bit rate encoding of the video;
• As with live encoding, select an appropriate window size and frame rate for the client’s access bandwidth
• Preroll can be a particular problem when pulling down a RealPix slide stream at the same time as a video stream. The server first estimates how much of the slide stream needs to be sent down beforehand in order to continue the stream without any interruptions. In a worse case scenario, it will cause the entire set of jpeg images to be downloaded before the video is started rather than pulling them down progressively as the stream plays. This can lead to an unacceptable preroll delay. To safeguard against this, don’t try to make the first slide appear at time zero in the playback timeline. Start at 10-15 seconds into the sequence. Also make sure that the interval between successive slides is longer than 15 seconds.

Click and Go Video

Using video streaming in a live teaching situation and linking in presentation slides and other Web material is not straightforward. Generation tools have yet to appear before this process becomes more automated and easier. The Click & Go Video project is devising a methodology for using video streaming in real time (lecture-based) and collaborative (tutorial and distance–based) learning. One of the technical underpinnings of the project is to look at automating the set up process using a Web-based SMIL generator. Figure 4 illustrates the type of options that may be presented to the user.

Figure 4: 'Click and Go' SMIL generator

Click and Go Video is being run under the JISC/DNER Programme: Enhancements for Learning and Teaching [14], and will evaluate its methodology with three teaching departments in Manchester - the Department of Textiles, UMIST (teaching of fashion marketing), Manchester Royal Infirmary (teaching of surgery) and the Department of Hospitality and Tourism, MMU (teaching of catering). Each of these case studies requires a different set of video production techniques and interaction for the learner covering:

• Real-time and on-demand streaming of lectures integrated with slides and video;
• Bookmarking key events in the video and providing hyperlinks to supporting learning material;
• Using streamed video together with desktop videoconferencing to support collaborative learning;
• Using micro video clips linked to specific practical activities.

Over 2001/2002, the project will be running a series of workshops on video streaming and producing ‘how to do it’ guides together with advice on learning and teaching best practice. Further details are available on the Click and Go Video Web site [15].

Viewpoint

In finishing this paper, the Advanced Telematics Centre has just completed another live Webcast, this time for the International Association of TEFL Conference in Brighton. The opening keynote speaker, Professor Carol Chapelle, from Iowa State University came in over an Internet videoconference link to Edinburgh University and from there over an ISDN6 link to the conference in Brighton, with the videoconference being picked up by Manchester University and streamed out over the Web. Despite the rather complex connection set up, the streaming worked. There were even email responses coming back to Brighton from people who were picking up the stream and who were able to have their questions relayed to the speaker. It was particularly fitting since the theme of the presentation was about the use of Web technology in TEFL. See [16] for the recorded session.

Streaming video technology has clearly reached a state where it can start to be used effectively. As the process of capturing and exporting media clips becomes easier and more straightforward, and set up tools become more widely available, then we will start to see a rapid take-up and expansion of its use. Take a look at the Scottish Parliament site [17]. This is really a vision of the sort of media enriched Web site that we can increasingly expect to see.

Acknowledgements

Many thanks to Paul Hammond-White from the Click and Go Video project, and from Jules Newgrosh and Fotis Zapheiropoulos at the Advanced Telematics Centre, for all their help and advice given in researching this paper.

References

1. The Future is Hybrid (Manchester event, February 2001): Multimedia Presentations, Ariadne and Jim Strom, 23-Mar-2001.
   URL: <http://www.ariadne.ac.uk/issue27/hl-multimedia/>
2. TERENA-NORDUnet Networking Conference 1999, Lund, Sweden.
   URL: <http://video.ldc.lu.se/terena_prog.htm>
3. Advanced Telematics Centre, University of Manchester.
   URL: <http://www.telematics.eu.org/>
4. Microsoft’s Windows Media Technologies: Streaming Methods – Web Server vs Streaming Media Server
   URL: < http://www.microsoft.com/windows/windowsmedia/en/compare/webservvstreamserv.asp>
5. Real Video Example, Antwerp clip, 56K
   URL: <http://www.telematics.eu.org/streaming/real56k.ram>
6. Real Video Example, Antwerp clip, ISDN
   URL: <http://www.telematics.eu.org/streaming/realisdn.ram>
7. Real Video Example, Antwerp clip, DSL
   URL: <http://www.telematics.eu.org/streaming/realdsl.ram>
8. Windows Media Example, Antwerp clip, 56K
   URL: <http://www.telematics.eu.org/streaming/windows56k.asx>
9. Windows Media Example, Antwerp clip, ISDN
   URL: <http://www.telematics.eu.org/streaming/windowsisdn.asx>
10. Windows Media Example, Antwerp clip, DSL
    URL: <http://www.telematics.eu.org/streaming/windowsdsl.asx>
11. Live Webcast Example: UMIST Student Fashion Show 2000
    URL: < http://khalibar.mcc.ac.uk:8080/ramgen/fashion2/fashion.smi>
12. Live Webcast Example: Mottram Millennium Festival
    URL: <http://khalibar.mcc.ac.uk:8080/ramgen/mottram2000/mottram.smi>
13. RealPix Example: xDSL Presentation by Patrick De Boeck from Telindus K-Net
    URL: <http://khalibar.mcc.ac.uk:8080/ramgen/telindus/telindus.smi>
14. JISC/DNER (Distributed National Electronic Resource)
    URL: <http://www.jisc.ac.uk/dner/>
15. Click and Go Video
    URL: <http://www.clickandgovideo.ac.uk/>
16. Keynote Speaker: Carol Chapelle, International TEFL Conference, Brighton, April 2001.
    URL: <http://khalibar.mcc.ac.uk:8080/ramgen/iatefl/iatefl.smi>
17. Scottish Parliament
    URL: <http://www.scottishparliamentlive.com/>

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Author Details

Jim Strom
Project Director
Advanced Telematics Centre
University of Manchester
MANCHESTER
M13 9PL
United Kingdom

jim.strom@man.ac.uk
< http://www.telematics.eu.org/>

Phone: +44 161 868 0545
Fax: +44 161 868 0565

Jim Strom is employed as a Team Leader for Telematics Applications at Manchester Computing, University of Manchester. His responsibilities cover management of the Advanced Telematics Centre, which provides advice and practical support to regional SMEs and public sector organisations in the use of interactive Internet technologies for business development and innovation. He is also the Project Director for the Click and Go Video project. Over the last 15 years he has been involved in university teaching for communications networks and distributed systems. Recent research covers Internet-based electronic commerce for SMEs in the local economy and the evaluation of IP-based desktop videoconferencing and streaming resources for deployment across city and regional network infrastructures.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

For citation purposes:
Strom, J. "Streaming Video: A Look Behind The Scenes", Cultivate Interactive, issue 4, 7 May 2001
URL: <http://www.cultivate-int.org/issue4/scenes/>

Copyright ©2000 - 2001 Cultivate. | Published by UKOLN | Design by ILRT | Contact Us