Despite my recent post about Ruby on Rails I'm pleased to announce that the first beta of Ruby.NET from my research lab. I'll let the release speak for itself:

We are pleased to announce the preliminary Beta release of the Gardens
Point Ruby.NET compiler. Note: this is not just a Ruby/.NET bridge, nor
a Ruby Interpreter implemented on .NET, but a true .NET compiler. The
compiler can be used to statically compile a Ruby source file into a
verifiable .NET v2.0 assembly or it can be used to directly execute a
Ruby source file (compile, load and execute). Our implementation is not
yet fully complete, but it is the only Ruby compiler that we know of for
either the .NET or JVM platforms that is able to pass all 871 tests in
the samples/test.rb installation test suite of Ruby 1.8.2.

Looking good. I haven't been involved in the actual development but I've been kept up to date with internal presentations and discussion and I'm currently looking after the website so I was involved in getting the release up this morning... but don't blame me if we get slashdotted.

I spent half of yesterday participating as a last minute ring-in for a Rails Day 2006 team. Going in to the competition I had a grand total of about 3 hours experience with Ruby so I was only going to be of limited help for the first few hours, but once I got up and running I have to say Ruby on Rails didn't impress me much. Here's some of my thoughts here. Please note that these are based on about 12 hours experience so I'd love people to point me in the direction of some counter-arguments.

My first impressions were actually quite positive. I particularly liked features like migrations for deploying database upgrades and it looks very easy to get the model up and running. The generators make basic editing pages quick and easy but about there is where I hit the wall.

My main issue is with the level of abstraction supplied by the views. Ruby on Rails still essentially uses the same abstraction level as previous systems like ASP, PHP and JSP. This means you basically place tags inside html files which spit out text. There's some functions supplied like link_to which slightly abstract away bits and pieces, but they're still just spitting out text.

ASP.NET on the other hand supplies controls which fully encapsulate a portion of UI. This means you can just set a few properties and let the control handle itself. e.g. if I want a paginated, editable grid you just pop a datagrid in and attach it to a datasource. I saw nothing built into Ruby on Rails which provided that type of abstraction. I've tried to work out how it could be done and I think it can be, but there's no assistance provided by the framework, and I haven't been able to work out how you might be able to do templated controls at all.

After all the hype surrounding Ruby on Rails I was disappointed by how much effort it took to set up views and associated controllers and particularly disappointed in how I always seemed to be considering the nitty gritty of HTML when generating views instead of just what info I wanted in the view and having the HTML abstracted away for me.

Like I said, I'm keen to have my knowledge corrected and/or improved so drop me a comment and some links if you can show me how to answer some of my issues.

It appears that my comment system was broken. I'm not sure how long for, but it's back now so if you had something to say and found you couldn't then please go back and try again.

Meanwhile, I'm planning on another couple of posts about my developments with regards to MPEG2 Transport Streams and programming BDA based DVB-T digital TV. I already have one user testing out the library and I've almost got recording working so I should have a useful .NET based BDA library soon.

Just a reminder that if you're interested in my non-technical posts (you know, the boring stuff about my life and such) then you need to go to Personal Blog rather than here.

Over the last weekend I've dived into some of the details of the MPEG-2 transport streams transmitted for Australian Digital TV with the goal of creating a C# library for scanning digital TV channels. This was largely prompted by a library released at forums.dvbowners.com which looked promising but which I wasn't 100% happy with due to licensing issues and its reliance on an extra DirectShow filter for parsing the MPEG tables. So instead I launched into my own library which performed the parsing itself. Surprisingly this wasn't particular difficult once you understand how the MPEG stream lays out the data. Unfortunately that info wasn't easily available anywhere... hence here's a summary of my findings with pointers to the full details. NOTE: I've only spent a couple of days gathering this info so don't expect it to be 100% correct - please leave corrections in the comments section.

I should be making the code available soon for others to play with so keep an eye on this site if you want to play.

Overview

First up, the MPEG stream contains a number of information tables for each frequency. When scanning for channels you simply tune into a certain frequency and gather information from these tables. Tables are identified using two identifiers - their PID (Packet ID) and TID (Table ID). There are quite a few tables involved, but the ones we're interested in for scanning channel info are these:

• Program Association Table (PAT)
  
• Program Map Table (PMT)
  
• Service Description Table (SDT)

The PAT is our first port of call. The PAT lists the services available for a certain frequency and is always available on PID 0x00. The PAT tells us two things about each service - its program number and a PID for finding its PMT. The PMT contains a list of all of the streams associated with a service.

The PMT gives us sufficient information for tuning into each channel, but it'd be nice to have the name of the channel. This is available from the SDT which is always available on PID 0x11. The SDT lists extra info for each service including the name of the service.

Many of these tables have a list of descriptors associated with their entries. There are quite a few different types of descriptors defined. For scanning we're particularly interested in the "Service Descriptor" available on the SDT which is where the service's name is found.

Where's the Details

Of course, this doesn't provide anywhere near enough information for actually implementing a scanner, but I can point you in the right direction for getting the details. The most important documents are:

• The MPEG-2 Transport Stream Specifications - ISO/IEC 13818-1 which a quick google discovered at http://neuron2.net/library/mpeg2/iso13818-1.pdf
  
• And the DVB-SI Specifications - ETSI EN 300 468 which I found at http://webapp.etsi.org/action/OP/OP20060428/en_300468v010701o.pdf

These will give you the layouts of all the tables and descriptors, but hopefully this article will speed up your search a little.

Implementing with DirectShow

DirectShow exposes these tables through the "MPEG-2 Sections and Tables" filter, using the IMpeg2Data interface. This interface takes a PID and a TID and returns the data for them. It should be relatively simple to find another tutorial on setting up a graph which contains this filter so I won't cover that.

Once you have the IMpeg2Data interface the following steps will perform a scan on the frequency that has been tuned into.

1. Obtain the PAT by requesting PID 0x00 and TID 0x00
   
2. Parse the PAT to discover the list of services available on this frequency. Store their program numbers and the PID for their PMT.
   
3. Request the PMT for each service using the PID given in the PAT and a TID of 0x02
   
4. Parse the PMT to gather the stream info including PIDs for each video & audio stream (and other streams such as TeleText).
   
5. Request the SDT through PID 0x11 and TID 0x42
   
6. Parse the SDT and search for Service Descriptors which contain the service names

Additionally you can request the Network Information Table (NIT) from PID 0x10, TID 0x40 to get the name of the provider of these services. The documentation indicates there should be a "Terrestrial Delivery System Descriptor" which could provide the actual centre frequency for the network. This could be slightly different to the frequency you originally tuned to, however so far I haven't been able to identify any of these in the actual broadcasts.

Future Work

EPG (Electronic Program Guide) data is also available from the Event Information Table (EIT) available at PID 0x12, TID 0x4E. I've done some basic tests with this and it's working, but I haven't finished cleaning it up.

Additionally, this work is only mildly attached to DVB-T - it should be pretty much identical for other DVB systems and somewhat close for ATSC based streams.