24 Trials of an accessible video player

Over the last week and a half I have been trialling the video player with different groups of students. The groups were trade students, students with English as a second language and students with a mild intellectual disability participating in a general work education class.

It was nice to leave the computer for a bit, talk to some real people and see them using the video player with ease.

The students were asked to watch some videos and after doing this they were then to try to “break” the video player. Luckily no-one succeeded. The students were later asked these questions:

• What do you think would make the video player better
  – there were no responses to this question other than “it was fine”, “it worked” and “it would be good if the video had little thumbnails so you can see the detail of what he is talking about.” (I pointed out that the second video actually had that – and this statement is about the video not the video player)
• What did you not like about the player?
  – again the responses were positive.
• Can you tell me what each of the buttons on the video player are for?
   – The common controls were identified easily. Some weren’t able to name the captions button but knew that it put up subtitles. No-one figured out what he audio description button did. I believe this is because they had no use for it (until later in the testing). One student suggested that the fast forward and backward buttons were skip one frame forward and back.
• Are there any other comments?
   – “The subtitles were good because I can read English better than I can understand it”

After the feedback I asked some students, approximately half of the students involved in the trial including the general work education class, to use the player pretending to be blind, deaf or have a motor skill disability.

For the students pretending to be blind I started the NVDA screen reader software and turned off their monitor so they “couldn’t see” anymore. I told them to use the Tab and enter keys for navigation. All students could navigate the entire page and video player. These students also gained an understanding of the importance of an audio description.

My “deaf” students had to turn down the sound on the computer/video player or take out their ear phones. These students used the captions to understand the content of the video. Some did need reminding to turn the captions on.

The remaining students with “poor motor skills” had to navigate the page and the video player without using a mouse. All could do this without issue.

I would have to say the testing of the video player went quite smoothly although three technical issues cropped up. Two quite small and fixable and one appearing to be a browser issue.

Issue one:

When trying to change the colour of the captions in full screen mode the Caption Settings dialog box is not keyboard accessible. I found this about ten minutes before the first session. I believe this has something to do with a variable being switched on while in full screen mode that makes it easier to navigate the video player. This variable needs to be switched off when the dialog opens.

Issue two:

Right clicking on the video itself reveals a menu that can be used to control the player, in Chrome at least. Selecting pause from the menu pauses the video but doesn’t switch the play-pause icon on the player itself. Not sure about that one…

Issue three:

Twice the video froze. The students and I could not restart the video without refreshing the page. I am not sure what the problem is although it is likely to be multiple mp4 videos running on a single Chrome web page. This issue was mentioned in a previous post.

23 Fullscreen video

One of the most enjoyable features of modern browsers is their ability of displaying page elements fullscreen. Video, for example, should have the option of playing fullscreen and not just being a box on the page.

Amazingly a fullscreen option, for me, has been very complex to produce for an accessible video player currently in development. This is for a few reasons:

• Fullscreen is reasonably new and initially it seems an easy concept though it becomes a little unclear how it works
• Most of the keyboard is disabled for security reasons
• If there is more than one video player on the web page how is the current video identified and then shown fullscreen
• While in fullscreen mode how can the current video player maintain keyboard focus. (While in fullscreen the keyboard still has access to the Tab key.)

How I think fullscreen works

Initially I thought that fullscreen might be a new window that appears with the content inside it. This would mean that fullscreen operated like a dialog, or popup, that is seen a lot on web pages. But this isn’t the case. Also fullscreen is definitely not modal. This means that keyboard focus would be contained within the element that is currently fullscreen and access to other elements on the page disabled.

The element running at fullscreen is not removed from the page tab order, nor is anything else on the page (which is currently hidden by the fullscreen element). To me this means that fullscreen clearly doesn’t make a new window or popup dialog.

Essentially the object running at fullscreen is zoomed into. It makes things bigger and removes the browser chrome. As the object is only zoomed into it is not removed from the page tab order. There are no new containers created that I know of or need to care about.

So something will need to happen to capture focus and make fullscreen function like a modal dialog box if fullscreen and accessible page elements is your goal.

Why are some of the keyboard keys disabled?

It’s believed that some, less than savoury developers, could use the fullscreen option to convince users that they are looking at their own computer screen while they are actually still on a malicious website. Then they could capture important personal information while the user is sure that they are in a secure place.

So this means that if I would like to use P for a play button - I can’t. Therefore the tab key is needed to access video player controls and other items that are running fullscreen.

NOTE: There are some ways of enabling the keyboard while in fullscreen though it is not consistently supported by browsers and in the end it wasn’t needed for this project.

Show the current video player fullscreen

This was the easiest part really, just need the order of events to fire at the right time. I identified the current video player on the mouse and key down events and accessed that information at a later point. Sometimes identifying an element on the up event is too late, or harder to sequence. This is especially the case when using a JavaScript library developed by others and you aren’t always sure when the event is firing.

NOTE: Did you know when the tab key is pressed that the next element is focused on when the key down event happens not on the key up event. Keyboards are different to mice. This helps a user move through the page quickly by holding down the tab key rather than pressing and releasing the key perhaps hundreds of times. This is true of all keys on the keyboard.

Maintaining keyboard focus

In the end the answer was easy. Thanks to Bronwyn Lapham.

Initially there was a lot of searching for answers to maintaining keyboard focus on or within an element and there were many trials and many errors. A method of turning off the tab indexes of elements not contained within in the current fullscreen video player was developed and worked reasonably well. My first programming teacher once described his programming on an example he put together as “a bit agricultural”. It gets the job done but isn’t very elegant. That was the code that I produced.

Bronwyn forwarded an article “Making an accessible dialog box” that I thought wouldn’t help because fullscreen doesn’t work like a dialog box; it zooms into an element.

While the article did specifically talk about ARIA roles and modality etc it also provided some code that I have not seen before although is apparently used by everyone. Finally I have caught up.

This following “.. technique was popularised by Peter-Paul Koch and is now in use by most JavaScript libraries” was used to create a tab enabled fullscreen video player. The code and more is explained thoroughly in the article above so I won’t do it again here and the adjustments were minor and dealt mainly with variable scope.

document.addEventListener("focus", function(event) {

    var dialog = document.getElementById("my-dialog");

    if (dialogOpen && !dialog.contains(event.target)) {

        event.stopPropagation();

        dialog.focus();

    }

}, true);

document.addEventListener("keydown", function(event) {

    if (dialogOpen && event.keyCode == 27) {

        // close the dialog

    }

}, true);

It is REALLY important to note that I needed to change the keydown event to keyup. I believe this is because while in fullscreen mode the escape key (27) on keydown only exits fullscreen mode. No other code seems to fire on that event – this is my assumption I didn’t challenge it to hard. Changing to keyup was successful because the player exited fullscreen on the keydown event. The escape key was now again available for the keyup event.

Thankyou Bronwyn (and Peter-Paul Koch), without this article I might have been able to do it but this was a much better way of getting everything to work. Not agricultural at all.

22 I think it works - an accessible HTML5 video player

So, I think, other than a lot of cleaning up, I have an accessible video player.

Here is what it does (so far??):

• Can access all functions with a keyboard
• Screen reader announces everything  - Though not always beautifully. This depends on the browser's accessibility implementation, ARIA support and how the screen reader itself behaves. Having said that you can find all of the controls
• No keyboard trap
• Is still accessible while in fullscreen mode
• Has a synched second audio track that can explain what is happening in the video to people who can't see it
• Captions of course - I'm reasonably happy with the implementation of them
• The captions can be presented in different:
• sizes
• fonts
• colours
• coloured background
• transparent background 0% 25% 50% 75% 100%
• The captions can be moved around the video so captions doesn't obscure anything
• Works on modern HTML5 browsers
• I've left the interface fairly dry, at least so far, so the developers can implement their own designs with CSS
• This project doesn't include a fallback to a flash version, their are plenty (?) of flash video players available and fallback was discussed in the first stage of this project.

But I still have that niggling feeling that something is missing... maybe it's just the jitters. What do you think? If it does what I say it does, is that enough?

Student testing will begin soon. We have had a small problem this year, as other projects may have also found, TAFE teachers who were teaching in 2012 aren't necessarily still available in 2013. I am in the process of organising a meeting with some teachers who I am quite sure will be very interested in trying out this player with their class. 

Luckily there are still some excellent teachers left.

21 Making code more robust

Ok, so this week sees me looking at the code and thinking that it could be a lot neater, more general and apply to more situations.  I find when I begin programming a new project I make my code very specific. I 'talk' to my objects directly by name rather than the better method of being general by referring to parents and children of them.

Being direct makes it easier, in my mind anyway, to get results quickly. This is because I know where I am going wrong when things aren't working for the path to the object is clear, or, clearly wrong. But lately I can see the flaw in this in starting this way.

It's no good to be specific with code that you may use more than once on a page. I have to control more than one video player for example. Therefore I will have to speak generally to all video players which is much more efficient than speaking to one at a time.

I've made a start hopefully it makes it better.

20 HTML5 video and Chrome’s one video per page problem

Being a web developer for nearly ten years has taught me to expect the unexpected when creating web pages for the many different browsers used around the world. CSS, accessibility, JavaScript and many other web technologies can’t be relied upon to produce the same results.

But this is a quirk that I didn’t expect – it isn’t possible to have more than one mp4 video on a page in the Chrome browser.

Perhaps we should expect that Chrome will not work well with mp4 video since moving to support the webm format at the beginning of 2011. http://blog.chromium.org/2011/01/html-video-codec-support-in-chrome.html. But Chrome does still play mp4 and in fact makes its default selection mp4 not webm.

Attempting to play multiple mp4 videos in Chrome causes these problems:

·         Chrome does not load one or more of the mp4 videos.

·         Causes Chrome to crash.

·         Will at best play only one video at a time.

The overarching problem is the state of HTML5 and browser implementation of it. When embedding video on web pages we are told current best practice is to have different video sources for different web browsers – mp4 for Safari, iOS, Android and IE and then  webm for Mozilla and Chrome (even though Chrome will still play the mp4). You may even elect to have Flash in there as well. A browser will then select the video it can play and will ignore the rest. It’s an elegant solution to the current issue of differing video format support in browsers. The big problem with this method is that Chrome defaults to the mp4 file. That is fine if only one video is on the page but causes serious problems when there are more.

So what is the answer, how is it fixed?

Use mp4 only in the video tag and show Chrome (and Firefox) the webm file. Chrome happily plays multiple webm files.

I know that seems counterintuitive after all that has just been written. I also know that Firefox, and of course Chrome, will be unable to play mp4 files effectively which is why we need a scripting fix.

JavaScript to the rescue

A JavaScript fix is needed, there isn’t anything else that can be done that I am aware of. Also keep in mind the two video file formats mp4 and webm are still required.

The video tag on your page should be (at a minimum) similar to:

<video width="640" height="360">

<source src="video/example.mp4" type="video/mp4" /> 

</video>

JavaScript, or in my case jQuery, is used to change the .mp4 file extension of the “src” to .webm when the page loads and the video/mp4 type attribute to video/webm when the page loads.

Below is the JavaScript (well jQuery) function that I have used: (the content argument in the function is the html that will be searched through looking for the video tag)

function SwapMP4forWEBM(content){

  if(jQuery.browser.chrome || jQuery.browser.mozilla){

    content.find("video").each(function(){

      $(this).find("source").attr("src", $(this).find("source").attr("src").replace(".mp4", ".webm"))

      $(this).find("source").attr("type", $(this).find("source").attr("type").replace("mp4", "webm"))

    })

  }

  return content.html()

}

That’s it.

Of course it is possible to do it the other way around; webm to mp4, but then you’ll just have to check for more browsers anyway. Added to this is the early iPad quirk of making sure that the mp4 file is the first one in the list of video files, otherwise it confuses the iPad and it can’t find the video to play. While the method really should work anyway on iPad(swapping webm for mp4) I haven’t tried it and why poke the tiger with a big stick if you don’t need to.

16 Converge 2012 eLearning Conference Presentation

Last week I presented a paper on the first stage of creating an accessible video player at the Converge 2012 eLearning Conference.

The discussion was about Stage 1 of this project, essentially providing captions for video, and focused on the topics:

• Captions must be easy for people to implement
• How are captions made available.

The second topic was easy enough; I went through the process of developing and implementing captions, it was quite dry and hopefully not overly technical.

After the presentation one of the delegates came up and started to discuss alternatives to the JavaScript used for Android and iOS device 'fixes'. He offered other alternatives that I thought were a bit more complex and would need some technical setup that mine didn't. He said that there was some JavaScript setup required in the alternative but he felt that my example already had JavaScript on the page so what was the difference. By it's mere presence it is perceived as too complex or technical.

I thought he had a point and I didn't have a reasonable answer other than any solution should produce/use  HTML5 'code' only with JavaScript in support. Using HTML5 on the page only will mean when HTML5 is fully implemented developers should only need to remove JavaScript from the page and not have to redevelop it.

When I thought about it more and looked at the JavaScript that was left in my example I couldn't see anything that the user/developer would need to change. This made me see my small mistake - there is no need for that JavaScript to be on the HTML page at all, it should be 'hidden' away in a linked file somewhere. Doing this removes all code from the page making it just that little bit easier. If it is visually easy to implement people won't be frightened off by it.

I need to remember this while working on this second stage of the project - get ALL of the code off the page if at all possible.

15 HTML5 video player - sync a second audio track

This week I am thinking about a second audio track for my video that provides audio descriptions for people with visual impairment.

To describe the problem that a second track would solve let's think about an actual classroom setting with a teacher demonstrating a task to a number of students. The teacher might say something like "Ok it is very important that you take this here and wrap it 5 times around this coil - but not the other coil. Before you turn it on make sure you press this button over here before flicking the power switch." Obviously the student can hear the teacher but might be stuck in the back of the classroom behind taller students and miss the visual information. If that is the case it is completely unclear what the teacher is talking about.

Many videos available online are excellent but every now and then slip into using non-descriptive words like 'this' and 'that'. If the user can't see what 'this' and 'that' are an otherwise excellent video can be rendered useless to them.

Early in 2011 Sylvia Pfeiffer at WHATWG and W3C were working on "... new HTML markup and a JavaScript interface for dealing with audio or video content that has more than just one audio and video track." Read the whole article: HTML5 multi-track audio or video.

This HTML markup should solve the problem of video with alternate tracks though it's not yet supported in current browsers. So what about now?

It turns out it isn't that hard to have a second audio track and to keep it relatively in sync with the video that is playing.

I have done it this way (basically):

1. Place some video on a HTML5 web page (using the video tag of course... not flash :-) )
2. Give the video tag an ID of videoTrack
3. Now add the audio track to the web page also (for my test I stripped the audio from the video so I could confirm if the audio was in fact in sync with the video)
4. Give the audio tag an ID of audioTrack
5. Create a function in JavaScript that looks like the one below:

function syncTracks(){

   var audio = document.getElementById("audioTrack"); 

   var video = document.getElementById("videoTrack");

   audio.currentTime = video.currentTime

}

Pretty much as easy as that, set the current time of the audio track to the current time of the video.  Essentially all that needs to happen now is to call the syncTracks() function whenever the video is played, paused or stopped.

NOTE: The video controls should also control the audio playback.

There is a big BUT!

The currentTime property of the video only seems to start updating when the video is running for a short time. I found that the audio track and the video gets out of sync by about a second or so depending on how the user interacts with play and pause buttons.

To circumvent this I created a timer which is called when play is pressed. It waits for about a tenth of a second and THEN calls the syncTracks() function. This works much better. The sound is only ever a small fraction of a second out using this method.

Here is the timer I used:

setTimeout(function(){syncTracks()},100)

While this is not perfect it is better than the alternative of waiting for browser support for multiple tracks.

13 Making an accessible HTML5 video player - first steps

Early attempts at making an accessible video player have been quite positive, buttons are working and are accessible with a keyboard and screen reader. There are no keyboard traps and the video player works in all HTML5 enabled browsers. Embedding the video player on the page is still quite simple and no special Javascript initialisation is required. There are no HTML5 tricks or work rounds required to get it to work.

So what have I learnt so far? I've learnt about fullscreen video playback.

An interesting point to note about video players in fullscreen.

Before this project started many different video players were reviewed. One in particular was found to be very accessible though not at all accessible in fullscreen mode. When the player was in fullscreen mode all controls disappeared and it was possible to tab to other items on the web page that were covered by the fullscreen video. This was very confusing but it is true of many video players and in fact many HTML items that are shown in fullscreen mode.

Problem number one for this project; how to turn off access to all other tab enabled HTML objects when the video is showing in fullscreen.

It turns out to be fairly easy, anything that has a CSS property of display:none is removed from the tab order.  Javascript is used to 'hide' all items that are not part of the video player in fullscreen mode and 'show' them when not in fullscreen mode. Currently JQuery's toggle function is doing this work.

The video controls that are part of your player also need to move depending on the current mode of the player. Currently the controls that are being used sit underneath the video when in normal mode but when the  player is set to fullscreen there is obviously no room for the controls to sit underneath. CSS is used to resize and move the controls when the video player enters and leaves fullscreen mode. Again the JQuery toggle function is used to achieve this.

I am sure that there are other ways of doing this and I leave this post thinking about my next internet search "Creating a modal window with Javascript'.

11 Thoughts on HTML5 video and subtitles

In due time, HTML5 will make the work of a web developer more efficient and effective. Once the final video format is chosen, embedding video in HTML5 webpages will become less complex.  In addition, the implementation of subtitles using the <track> tag will eventually require less or no JavaScript fixes.

In the meantime, a Flash fall-back will be necessary as IE8 is expected to be used widely until 2015 when Microsoft stops support for Windows XP and it currently does not support the <video> tag.

There is still some degree of complexity in creating subtitles, in that there is a lack of available software for easy and less time consuming file conversion to HTML5 compatible formats. However, this may change at some stage as the available features in Universal Subtitles/Amara were better than expected and have the potential to be developed further.

Google are currently working on providing subtitles in YouTube by developing voice recognition. However this needs further refinement as when voice recognition was tested with video accompanied by good quality audio; approximately 10% of what was said was correct. Discussion of these results can be viewed here: http://www.youtube.com/watch?v=krq1msguNFo.

It was not possible to locate information that indicated how iOS or Android will support HTML5/web based subtitles in the future. This was a little disappointing as these devices seem to be behind the rapid uptake of HTML5 among web developers. Perhaps changes will occur when the HTML5 specification is completed.

Currently there is a lack of tools that will create, or help to create, WebVTT format files. It is expected that this will change as the format is very similar to the fairly common SRT format and may not require major changes to existing software. Over the life of the project it has been noted that changes were still happening and there was an increase in available tools. (Such is the rate of change that the information in this document was frequently changed to include new developments.)

BubbleJS http://bubbles.childnodes.com/ provides a promising future for both triggering subtitles successfully and providing a high functioning video player. The BubbleJS player can trigger actions that happen outside of the video player such as opening, closing, animating, effecting colour changes etc. on other html objects. This is an excellent result for education in that it provides a video player with accessible video that interacts with other page elements to present information related to the video, in engaging and innovative ways. Hopefully the setup of BubbleJS becomes easier.

Working with video on the web has, over the years, become much easier and a lot of time savings can be found in good processes and work practices. Hopefully the time saved can be dedicated to the creation of subtitles and accessible video for all internet users.

Related posts:

Accessible HTML5 video for all
How to create a basic HTML5 webpage
Fall back to Flash
Current HTML5 subtitle support
Subtitle creation process
Link subtitles to video
Making subtitles work now
A Flash solution
Making it work now, on an Android device
Making it work now, on an Apple device
Thoughts on HTML5 video and subtitles - conclusion

09 Making it work now, on an Android device

Androids are more complex. The video opens in a native Android player and does not play in the webpage directly because the subtitle track is ignored by the internal Android video player. While it is possible to embed subtitles on the video (as described in the post Making It Work Now on an Apple Device) the current Android video player does not give the option of selecting them.

Therefore a Flash alternative that does supply subtitles is required. To supply the Flash alternative a Flash fall-back in the HTML5 <video> tag is needed in combination with a JavaScript fix that:

• removes the <video> tag completely (which, as a by-product removes the HTML5 Flash fall-back)
• re-embeds the Flash video.

In order to implement HTML5 and video in the easiest most conformant and future proof way, two JavaScript libraries were used;  SWFObject and jQuery, and a short customised script.


Note: Custom JavaScript could have been written to do everything mentioned above though it is easier to implement the proposed fixes using these common JavaScript libraries. JQuery is becoming more widely used and it is not unlikely that it is already available on the page that will hold the video that requires this fix.

Add the required JavaScript to the page

Step 1:
Download the required JavaScript libraries SWFObject and jQuery.

• SWFObject: http://code.google.com/p/swfobject/
• jQuery: http://jquery.com/

At the time of writing the current versions are SWFObject2.2 and jQuery 1.7.2 use the latest versions that you can find to ensure cross browser compatibility.

Step 2:

• Place the JavaScript files into the ‘website’ folder*.

Note*: When the SWFObject folder is unzipped it creates a number of folders that contain examples of how to use it and 2 SWFObject JavaScript files. Both files will work though the one file that is used in the examples below is the minified version found in the folder: swfobject_2_2/swfobject/swfobject.js.

• The files now in the ‘website’ folder should appear as below:
• videoplayer.fla
• videoplayer.swf
• SkinUnderPlayStopSeekCaptionVol.swf
• html5videoex.mp4
• html5videoex.webm
• html5video.htm
• subs.vtt
• subs.xml
• playr.js
• playr.css
• images (folder)
• jquery-1.7.2.min.js (or similar depending on current version)
• swfobject.js (or similar depending on current version)

Links to the downloaded JavaScript files, videos and website folder can be found at the bottom of the page.


Step 3:

• Open html5video.htm in a preferred web editor
• Paste the following HTML into the <head> of the document above the playr.js link. Ensure that the same order as below is kept as it is important.

<link rel="stylesheet" type="text/css" href="playr.css" />
<script src="jquery-1.7.2.min.js" type="text/javascript" language="javascript"></script>
<script src="swfobject.js" type="text/javascript" language="javascript"></script>
<script src="playr.js" type="text/javascript"></script>


Step 4:
This step will:

• include some custom code that inserts a flash object
• use information from the flash fall-back in the video tag
• remove the <video> tag completely
• complete the process on an Android device only.

1. Insert the code below between the swfobject.js and playr.js <script> tags.

The code below waits until the page is ready. Once the page is ready it looks for the Android* operating system (OS) and if it finds the Android OS it:

• creates a <div> for the new Flash to be embedded in
• embeds the Flash video into the newly created <div> using information that is obtained from the Flash fall-back in the <video> tag
• deletes the <video> tag from the page as it is no longer required

<script src="swfobject.js" type="text/javascript" language="javascript"></script>
<script type="text/javascript">
$(document).ready(function(){

if(navigator.userAgent.toLowerCase().indexOf("android") > -1) {

$("<div id='flashContent'></div>").insertBefore("video");
swfobject.embedSWF($("video object").attr("data")+”?”, "flashContent", $("video object").attr("width"), $("video object").attr("height"), "9.0.0");  //all on one line.
$("video").remove();

}

})
</script>
<script src="playr.js" type="text/javascript"></script>



IMPORTANT NOTE*: For some reason Flash video does not work on Android in a Flash video player **unless there is a question mark (?) after the path to the Flash file. A question mark is used to send information from the web page back into Flash. This is why there is a question mark in the code above - ($("video object").attr("data")+”?”, – If variables are being sent to Flash from the standard Flash embed created in the HTML5 <video> tags it is wise to remove the question mark from the JavaScript code. Here is the amended code - ($("video object").attr("data"),


NOTE**: playr.js still runs though it does not find a <video> tag and does not initialize any code that causes confusion for the Android device.


Step 5:

Upload the website and test this on an Android device.

Related posts:

Accessible HTML5 video for all
How to create a basic HTML5 webpage
Fall back to Flash
Current HTML5 subtitle support
Subtitle creation process
Link subtitles to video
Making subtitles work now
A Flash solution
Making it work now, on an Android device
Making it work now, on an Apple device

Downloads

• website folder
• html5videoex.mp4
• html5videoex.webm

08 A Flash solution

Older browsers that don’t support HTML5 video need to use the Flash fall-back as described earlier for IE 6,7 and 8. Firstly the Flash video player needs to be altered to show subtitles. The steps below will update the videoplayer.fla and videoplayer.swf files.

There is a video demonstration of the steps below available here:
http://www.youtube.com/watch?v=HcG3L1P149U

Steps	Comments
Open videoplayer.fla that was created earlier.	This can be found in the ‘website’ folder.
Select the Text Tool from the toolbar – don’t draw a text box just yet.	Setting the options before drawing the text box seemed to make a better result in Adobe Flash CS5. This is probably not important though it was frustrating.
Set these properties before drawing a text box: Make the text box a Dynamic text box Choose a readable font such as Arial Set the font size to 16pt Choose a bright colour such as white or yellow Click on the ‘Character Embedding’ button Choose ‘Uppercase’, ‘Lowercase’, ‘Numerals’, Punctuation’ and ‘Basic Latin’	When the text box, or text tool, is selected all of the text box properties can be located in the Properties panel.
Draw a text box at the bottom of the video player screen.	Place it where subtitles usually appear.
Give the text box an instance name of subsText.	When the text box is selected there is an ‘Instance Name’ option located in the Properties panel.
From the ‘Window’ menu choose ‘Components’.
Drag the FLVPlaybackCaptioning component to the left of the video.	The positioning does not matter as the component is not visible when the movie is running; placing it off to the side of the video player keeps the work area clear.
Close the ‘Components’ window.
From the ‘Window’ menu choose ‘Component Inspector’.
Make sure that the FLVPlaybackCaptioning component is selected.	Click on the component with the ‘Selection’ tool (Black Arrow)
Select the ‘Parameters’ tab on the ‘Component Inspector’ window.
Set the CaptionTargetName property to subsText	This is the name of the text box created earlier.
Set the Source property to subs.xml	subs.xml is the subtitle file created for Flash earlier.
From the menu choose ‘Control’ and then ‘Test Movie’.	The subtitles should appear though they may be a little flat and not stand out as much as expected. One way to fix this is to attach a Drop Shadow filter to the text box.
Select the text box.
From the ‘Properties’ panel choose ‘Filters’.
Select the ‘Add Filter’ button at the bottom left of the panel.	The icon looks like a piece of paper.
Choose Drop Shadow.
Save and test the movie again.
A Flash file that plays subtitles has been created.	This should have updated the videoplayer.swf file that is in the ‘Website’ folder.
Go back to the ‘Website’ folder and open the html5video.htm  file in IE 6,7,8 to view the Flash fall-back. The subtitles should now be working there.

• Go back to the ‘Website’ folder and open the html5video.htm  file in IE 6,7,8 to view the Flash fall-back. The subtitles should now be working there.

Related posts:

Accessible HTML5 video for all
How to create a basic HTML5 webpage
Fall back to Flash
Current HTML5 subtitle support
Subtitle creation process
Link subtitles to video
Making subtitles work now
A Flash solution
Making it work now, on an Android device
Making it work now, on an Apple device
Thoughts on HTML5 video and subtitles - conclusion

Downloads

• website folder
• html5videoex.mp4
• html5videoex.webm

07 Making subtitles work now

Making the <track> tag work in modern web browsers

At the time of writing there are three JavaScript solutions that offer the required subtitle functionality to the <video> tag; Playr.js, MediaElement and BubbleJS.

Playr.js
The chosen JavaScript solution for this project is playr.js.(Instructions on downloading and using playr.js are outlined in Downloading and Using Playr.js)


Media Element
MediaElement.js http://mediaelementjs.com/, appears to be a robust alternative though the configuration is more complex than other available options.

BubbleJS
BubbleJS http://bubbles.childnodes.com/  is well designed and has the most potential for the creation of interesting video interaction in the future. The downside to this however is that it has some features which make it less compliant and easy to use. For example:

• it only supports .srt subtitles
• it has a complex setup which makes it difficult for beginners
• the subtitle track is linked through JavaScript not by the <track> tag in the HTML.

Downloading and using playr.js

Playr.js is downloadable here: https://github.com/delphiki/Playr

Step 1:

• Download the repository as a zip file by selecting the ZIP icon

• Unzip the files and then place them into the ‘website’ folder created earlier.
• Delete the files called CHANGELOG.markdown and README.markdown(  they are not important for implementing subtitles).

Your website folder should now include these files and folders:

• videoplayer.fla
• videoplayer.swf
• SkinUnderPlayStopSeekCaptionVol.swf
• html5videoex.mp4
• html5videoex.webm
• html5video.htm
• subs.vtt
• subs.xml
• playr.js
• playr.css
• images (folder)

Links to the downloaded JavaScript files, videos and website folder can be found at the bottom of the page.


Step 2:
Now link the required files to the webpage.

• Change the <head> of the HTML to appear as below:  (This will link the required JavaScript to the webpage.)

<head>
<title>My HTML5 Video Page</title>
<link rel="stylesheet" type="text/css" href="playr.css" />
<script src="playr.js" type="text/javascript"></script>
</head>



Step 3:
To enable the JavaScript a small change needs to be made to the <video> tag.

• Update the <video> tag so that it matches below:

<video class="playr_video" width="640" height="360" controls="controls">

• Save and test the web page in a modern browser.

Subtitles should appear on the page over the video, if they do not, use the video controls that appear underneath the video to show the captions.*

NOTE: * There is a current issue (April 2012) with Google Chrome where the subtitles will not appear unless the video is being viewed over the internet. Local computer testing does not work. Either upload the ‘website’ folder to the internet or test using another browser such as Firefox.

Related posts:

Accessible HTML5 video for all
How to create a basic HTML5 webpage
Fall back to Flash
Current HTML5 subtitle support
Subtitle creation process
Link subtitles to video
Making subtitles work now
A Flash solution
Making it work now, on an Android device
Making it work now, on an Apple device
Thoughts on HTML5 video and subtitles - conclusion

Downloads

• website folder
• html5videoex.mp4
• html5videoex.webm

06 Link subtitles to video

Once you have created and saved the subtitles they have to be linked to the video on the webpage. This post will discuss:

Embedding subtitles using HTML5 only
The section ‘Embedding Subtitles using HTML5 only’ will describe how subtitles will be provided to HTML5 based websites in the future.

Before starting

• Move the subs.srt, subs.vtt and the subs.xml file to the ‘website’ folder that was created earlier

All files required should now be in the same folder, encoded video files, HTML5 web pages and the subtitle files. It is important to ensure that all sub title files are available in the ‘website’ folder. This will make files ready to work with and link to the HTML5 encoded video.
The website folder should contain these files:

• videoplayer.fla
• videoplayer.swf
• SkinUnderPlayStopSeekCaptionVol.swf
• html5videoex.mp4
• html5videoex.webm
• html5video.htm
• subs.srt
• subs.vtt
• subs.xml

Links to the completed subtitles, videos and website folder can be found at the bottom of the page.

Embed subtitles using HTML5 only

These instructions will provide the HTML5 code for embedding subtitles. Before starting, open the html5video.htm webpage that was created earlier. That web page contained 3 links to 2 videos:

• html5videoex.mp4
• html5videoex.webm

Two of the links in this page were for HTML5 enabled devices and the other was for the Flash based fall back if the browser didn’t support HTML5. The HTML code in this webpage currently appears as:

Current code


<body>

<video width="640" height="360" controls="controls">

<source src="html5videoex.mp4" type="video/mp4" />
<source src="html5videoex.webm" type="video/webm" />
<object id="flashContent" type="application/x-shockwave-flash" data="videoplayer.swf" width="640" height="400">

<param name="movie" value="videoplayer.swf" />
<p>Download the Flash plugin from <a href="http://www.adobe.com" title="Download Flash plugin">www.adobe.com</a></p>

</object>

</video>

</body>



Updated code
Add the line of code below after the <source> tags and before the <object> tag to create the link to the subtitle:

<track src="subs.vtt" kind="subtitles" srclang="en" label="English subtitles" default />

The <track> tag is all that is required to meet the proposed HTML5 specification of providing subtitles.
Currently there are no browsers that render the subtitles from the <track> tag only and therefore a JavaScript solution must be provided. This is discussed in the ‘Making Subtitles Work Now’ post (The link to this can be found at the end of this page).

src – this is the location and name of the subtitle file
kind – specifies whether captions, subtitles, chapters, descriptions or metadata are being provided.
srclang – the language of the track
label – the title of the track
default – specifies that the track is enabled by default. Many track elements can be used for the same video.

The webpage code now with the <track> tag included appears as below:

<body>

<video width="640" height="360" controls="controls">

<source src="html5videoex.mp4" type="video/mp4" />
<source src="html5videoex.webm" type="video/webm" /> 

<track src="subs.vtt" kind="subtitles" srclang="en" label="English subtitles" default />

<object id="flashContent" type="application/x-shockwave-flash" data="videoplayer.swf" width="640" height="400">
<param name="movie" value="videoplayer.swf" />
<p>Download the Flash plugin from <a href="http://www.adobe.com" title="Download Flash plugin">www.adobe.com</a></p>

</object>

</video>

</body>

1. Save the changes to the web page and test the result in a browser. In future versions of web browsers this is all that will be needed to get subtitles to work. As of April 2012* there are no browsers that support the <track> tag.

NOTE*: Web Developers should make sure that the sites and pages that are developed, work on all commonly used web browsers including old and new. It is expected that IE8, that does not have <video> tag support will be in common use for the next few years, 2015 and beyond. Therefore while the <track> tag will start to become more widely incorporated into future browsers web developers will still need to make sure that subtitles are available in older browsers that don’t support <track>.

Related posts:

Accessible HTML5 video for all
How to create a basic HTML5 webpage
Fall back to Flash
Current HTML5 subtitle support
Subtitle creation process
Link subtitles to video
Making subtitles work now
A Flash solution
Making it work now, on an Android device
Making it work now, on an Apple device
Thoughts on HTML5 video and subtitles - conclusion

Downloads

• website folder
• html5videoex.mp4
• html5videoex.webm

01 Accessible HTML5 video for all

What is this project about

This series of posts have been developed as a result of a project undertaken by Learning Development that explored different ways video and subtitles could be incorporated into a HTML5 based website.

The video was first incorporated into the website, then tested on new and old browsers and on mobile devices such as iPad, iPod, Android Phone and an Android tablet. The testing was designed to ensure that video was visible and usable in all new and old browsers and did not lack accessibility features such as subtitles.

The posts will provide advice and instructions on how to incorporate video into a HTML5 based website. They will:

• discuss video formats required for HTML5 video
• discuss subtitle formats and use in HTML5
• identify some compression software that may help produce video in the right formats*

It is also hoped that the advice and instructions will not be overly complex and that only minor changes to the methods described in the posts will be required once HTML5 specification is fully accepted.

*Note: The posts discuss compression formats but do not cover video compression techniques in great detail.

What is HTML and HTML5?

HTML is a code language which is used to develop webpages. It is the code that makes webpages look and act the way they do. For example, HTML coding will tell the webpage what text colours to use, what to do when the user clicks on a link, when to play a video, or what happens when the user rolls the mouse over an image. HTML5 is the name given to the updated HTML standard.

Almost every web page on the internet is made with, or incorporates, HTML. HTML was developed initially in 1990 by Tim Berners-Lee and was last updated to its current standard in 2000. However, since the introduction of mobile devices there has been a need to adapt the current HTML standard to suit these emerging technologies.  For example, HTML is now being upgraded to include interactive features such as video and audio which can be viewed without 3rd party plugins.

Although HTML5 is still in the process of developing, it is expected that HTML5 will soon replace current HTML technology and many developers and websites are making the transition now.

Is HTML5 widely supported?

To add a video into an HTML5 webpage, the tag <video>is added to the coding.

All new browsers support the <video> tag and some or most of the recommended HTML5 features.

However, as HTML5 is currently under development it may be necessary to provide multiple video formats to be sure that the video will display in both old and new browsers (required video formats are discussed in the section Video Formats below).

To see how well your browser supports HTML5 visit http://html5test.com/.

Watching video on the web

The current HTML standard doesn’t support video being played directly from webpages. Therefore, in order to view videos attached to webpages, the user is required to download 3rd party plugins such as Flash or QuickTime.

As 3rd party plugins, such as Flash, can drain the battery of mobile devices and affect performance overall, both Apple and Microsoft have made the decision to support the development of HTML5 as a way of replacing the need to download plugins*. In addition to this Adobe, the developer of the Flash player, have announced that it will no longer update its mobile Flash player and that it fully supports the proposed HTML5 standard.

*See Steve Jobs’ article ‘Thoughts on Flash’ http://www.apple.com/hotnews/thoughts-on-Flash/.

How to put video on my HTML5 webpage

Video formats

Currently, as of April 2012, HTML5 has not confirmed which video formats will be supported. In the meantime, to make sure all users of a site can see the video on any computer or mobile device, videos should be supplied in multiple formats. For example, the formats that must be offered at a minimum are:

• MP4 (H.264)
• WebM
• OGG was used by earlier versions of Firefox and should be considered as a format still worth supplying

How HTML5 works with multiple formats


The HTML5 <video> tag works using a fall back method.  This means that a browser that cannot view one video format, will fall back to using the next in the queue. If the browser still does not understand the video format it can keep looking through the <video> tag for more information or a format that it does understand.

Some very old browsers don’t support the HTML5 video tag at all. In this case another format that can be offered is the FLV file type.  Flash also supports the MP4 video codec so it is not overly important to supply the FLV format type as well. The MP4 video must be H.264 encoded to work with a Flash video player.

Related posts:

Accessible HTML5 video for all
How to create a basic HTML5 webpage
Fall back to Flash
Current HTML5 subtitle support
Subtitle creation process
Link subtitles to video
Making subtitles work now
A Flash solution
Making it work now, on an Android device
Making it work now, on an Apple device
Thoughts on HTML5 video and subtitles - conclusion