You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
454 lines
20 KiB
454 lines
20 KiB
<chapter> <title> The complex multi-layer input </title>
|
|
|
|
<para>
|
|
The idea behind the input module is to treat packets, without knowing
|
|
at all what is in it. It only takes a packet,
|
|
reads its ID, and delivers it to the decoder at the right time
|
|
indicated in the packet header (SCR and PCR fields in MPEG).
|
|
All the basic browsing operations are implemented without peeking at the
|
|
content of the elementary stream.
|
|
</para>
|
|
|
|
<para>
|
|
Thus it remains very generic. This also means you can't do stuff like
|
|
"play 3 frames now" or "move forward 10 frames" or "play as fast as you
|
|
can but play all frames". It doesn't even know what a "frame" is. There
|
|
is no privileged elementary stream, like the video one could be (for
|
|
the simple reason that, according to MPEG, a stream may contain
|
|
several video ES).
|
|
</para>
|
|
|
|
<sect1> <title> What happens to a file </title>
|
|
|
|
<para>
|
|
An input thread is spawned for every file read. Indeed, input structures
|
|
and decoders need to be reinitialized because the specificities of
|
|
the stream may be different. <function> input_CreateThread </function>
|
|
is called by the interface thread (playlist module).
|
|
</para>
|
|
|
|
<para>
|
|
At first, an input plug-in capable of reading the plugin item is looked
|
|
for [this is inappropriate : we should first open the socket,
|
|
and then probe the beginning of the stream to see which plug-in can read
|
|
it]. The socket is opened by either <function> input_FileOpen</function>,
|
|
<function> input_NetworkOpen</function>, or <function>
|
|
input_DvdOpen</function>. This function sets two very important parameters :
|
|
<parameter> b_pace_control </parameter> and <parameter> b_seekable
|
|
</parameter> (see next section).
|
|
</para>
|
|
|
|
<note> <para>
|
|
We could use so-called "access" plugins for this whole mechanism
|
|
of opening the input socket. This is not the case because we
|
|
thought only those three methods were to be used at present,
|
|
and if we need others we can still build them in.
|
|
</para> </note>
|
|
|
|
<para>
|
|
Now we can launch the input plugin's <function> pf_init </function>
|
|
function, and an endless loop doing <function> pf_read </function>
|
|
and <function> pf_demux</function>. The plugin is responsible
|
|
for initializing the stream structures
|
|
(<parameter>p_input->stream</parameter>), managing packet buffers,
|
|
reading packets and demultiplex them. But in most tasks it will
|
|
be assisted by functions from the advanced input API (c). That is
|
|
what we will study in the coming sections !
|
|
</para>
|
|
|
|
</sect1>
|
|
|
|
<sect1> <title> Stream Management </title>
|
|
|
|
<para>
|
|
The function which has opened the input socket must specify two
|
|
properties about it :
|
|
</para>
|
|
|
|
<orderedlist>
|
|
<listitem> <para> <emphasis> p_input->stream.b_pace_control
|
|
</emphasis> : Whether or not the stream can be read at our own
|
|
pace (determined by the stream's frequency and
|
|
the host computer's system clock). For instance a file or a pipe
|
|
(including TCP/IP connections) can be read at our pace, if we don't
|
|
read fast enough, the other end of the pipe will just block on a
|
|
<function> write() </function> operation. On the contrary, UDP
|
|
streaming (such as the one used by VideoLAN Server) is done at
|
|
the server's pace, and if we don't read fast enough, packets will
|
|
simply be lost when the kernel's buffer is full. So the drift
|
|
introduced by the server's clock must be regularly compensated.
|
|
This property controls the clock management, and whether
|
|
or not fast forward and slow motion can be done.</para>
|
|
|
|
<note> <title> Subtilities in the clock management </title> <para>
|
|
With a UDP socket and a distant server, the drift is not
|
|
negligible because on a whole movie it can account for
|
|
seconds if one of the clocks is slightly fucked up. That means
|
|
that presentation dates given by the input thread may be
|
|
out of sync, to some extent, with the frequencies given in
|
|
every Elementary Stream. Output threads (and, anecdotically,
|
|
decoder threads) must deal with it. </para>
|
|
|
|
<para> The same kind of problems may happen when reading from
|
|
a device (like video4linux's <filename> /dev/video </filename>)
|
|
connected for instance to a video encoding board.
|
|
There is no way we could differentiate
|
|
it from a simple <command> cat foo.mpg | vlc - </command>, which
|
|
doesn't imply any clock problem. So the Right Thing (c) would be
|
|
to ask the user about the value of <parameter> b_pace_control
|
|
</parameter>, but nobody would understand what it means (you are
|
|
not the dumbest person on Earth, and obviously you have read this
|
|
paragraph several times to understand it :-). Anyway,
|
|
the drift should be negligible since the board would share the
|
|
same clock as the CPU, so we chose to neglect it. </para> </note>
|
|
</listitem>
|
|
|
|
<listitem> <para> <emphasis> p_input->stream.b_seekable
|
|
</emphasis> : Whether we can do <function> lseek() </function>
|
|
calls on the file descriptor or not. Basically whether we can
|
|
jump anywhere in the stream (and thus display a scrollbar) or
|
|
if we can only read one byte after the other. This has less impact
|
|
on the stream management than the previous item, but it
|
|
is not redundant, because for instance
|
|
<command> cat foo.mpg | vlc - </command> is b_pace_control = 1
|
|
but b_seekable = 0. On the contrary, you cannot have
|
|
b_pace_control = 0 along with b_seekable = 1. If a stream is seekable,
|
|
<parameter> p_input->stream.p_selected_area->i_size </parameter>
|
|
must be set (in an arbitrary unit, for instance bytes, but it
|
|
must be the same as p_input->i_tell which indicates the byte
|
|
we are currently reading from the stream).</para>
|
|
|
|
<note> <title> Offset to time conversions </title> <para>
|
|
Functions managing clocks are located in <filename>
|
|
src/input/input_clock.c</filename>. All we know about a file
|
|
is its start offset and its end offset
|
|
(<parameter>p_input->stream.p_selected_area->i_size</parameter>),
|
|
currently in bytes, but it could be plugin-dependant. So
|
|
how the hell can we display in the interface a time in seconds ?
|
|
Well, we cheat. PS streams have a <parameter> mux_rate </parameter>
|
|
property which indicates how many bytes we should read in
|
|
a second. This is subject to change at any time, but practically
|
|
it is a constant for all streams we know. So we use it to
|
|
determine time offsets. </para> </note> </listitem>
|
|
</orderedlist>
|
|
|
|
</sect1>
|
|
|
|
<sect1> <title> Structures exported to the interface </title>
|
|
|
|
<para>
|
|
Let's focus on the communication API between the input module and the
|
|
interface. The most important file is <filename> include/input_ext-intf.h,
|
|
</filename> which you should know almost by heart. This file defines
|
|
the input_thread_t structure, the stream_descriptor_t and all programs
|
|
and ES descriptors included (you can view it as a tree).
|
|
</para>
|
|
|
|
<para>
|
|
First, note that the input_thread_t structure features two <type> void *
|
|
</type> pointers, <parameter> p_method_data </parameter> and <parameter>
|
|
p_plugin_data</parameter>, which you can respectivly use for buffer
|
|
management data and plugin data.
|
|
</para>
|
|
|
|
<para>
|
|
Second, a stream description is stored in a tree featuring program
|
|
descriptors, which themselves contain several elementary stream
|
|
descriptors. For those of you who don't know all MPEG concepts, an
|
|
elementary stream, aka ES, is a continuous stream of video or
|
|
(exclusive) audio data, directly readable by a decoder, without
|
|
decapsulation.
|
|
</para>
|
|
|
|
<para>
|
|
This tree structure is illustrated by the following
|
|
figure, where one stream holds two programs.
|
|
In most cases there will only be one program (to my
|
|
knowledge only TS streams can carry several programs, for instance
|
|
a movie and a football game at the same time - this is adequate
|
|
for satellite and cable broadcasting).
|
|
</para>
|
|
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="stream.png" format="PNG" scalefit="1" scale="80"/>
|
|
</imageobject>
|
|
<imageobject>
|
|
<imagedata fileref="stream.gif" format="GIF" />
|
|
</imageobject>
|
|
<textobject>
|
|
<phrase> The program tree </phrase>
|
|
</textobject>
|
|
<caption>
|
|
<para> <emphasis> p_input->stream </emphasis> :
|
|
The stream, programs and elementary streams can be viewed as a tree.
|
|
</para>
|
|
</caption>
|
|
</mediaobject>
|
|
|
|
<warning> <para>
|
|
For all modifications and accesses to the <parameter>p_input->stream
|
|
</parameter> structure, you <emphasis>must</emphasis> hold
|
|
the p_input->stream.stream_lock.
|
|
</para> </warning>
|
|
|
|
<para>
|
|
ES are described by an ID (the ID the appropriate demultiplexer will
|
|
look for), a <parameter> stream_id </parameter> (the real MPEG stream
|
|
ID), a type (defined
|
|
in ISO/IEC 13818-1 table 2-29) and a litteral description. It also
|
|
contains context information for the demultiplexer, and decoder
|
|
information <parameter> p_decoder_fifo </parameter> we will talk
|
|
about in the next chapter. If the stream you want to read is not an
|
|
MPEG system layer (for instance AVI or RTP), a specific demultiplexer
|
|
will have to be written. In that case, if you need to carry additional
|
|
information, you can use <type> void * </type> <parameter> p_demux_data
|
|
</parameter> at your convenience. It will be automatically freed on
|
|
shutdown.
|
|
</para>
|
|
|
|
<note> <title> Why ID and not use the plain MPEG <parameter>
|
|
stream_id </parameter> ? </title> <para>
|
|
When a packet (be it a TS packet, PS packet, or whatever) is read,
|
|
the appropriate demultiplexer will look for an ID in the packet, find the
|
|
relevant elementary stream, and demultiplex it if the user selected it.
|
|
In case of TS packets, the only information we have is the
|
|
ES PID, so the reference ID we keep is the PID. PID don't exist
|
|
in PS streams, so we have to invent one. It is of course based on
|
|
the <parameter> stream_id </parameter> found in all PS packets,
|
|
but it is not enough, since private streams (ie. AC3, SPU and
|
|
LPCM) all share the same <parameter> stream_id </parameter>
|
|
(<constant>0xBD</constant>). In that case the first byte of the
|
|
PES payload is a stream private ID, so we combine this with
|
|
the stream_id to get our ID (if you did not understand everything,
|
|
it isn't very important - just remember we used our brains
|
|
before writing the code :-).
|
|
</para> </note>
|
|
|
|
<para>
|
|
The stream, program and ES structures are filled in by the plugin's
|
|
<function> pf_init()
|
|
</function> using functions in <filename> src/input/input_programs.c,
|
|
</filename> but are subject to change at any time. The DVD plugin
|
|
parses .ifo files to know which ES are in the stream; the TS plugin
|
|
reads the PAT and PMT structures in the stream; the PS plugin can
|
|
either parse the PSM structure (but it is rarely present), or build
|
|
the tree "on the fly" by pre-parsing the first megabyte of data.
|
|
</para>
|
|
|
|
<warning> <para>
|
|
In most cases we need to pre-parse (that is, read the first MB of data,
|
|
and go back to the beginning) a PS stream, because the PSM (Program
|
|
Stream Map) structure is almost never present. This is not appropriate,
|
|
though, but we don't have the choice. A few problems will arise. First,
|
|
non-seekable streams cannot be pre-parsed, so the ES tree will be
|
|
built on the fly. Second, if a new elementary stream starts after the
|
|
first MB of data (for instance a subtitle track won't show up
|
|
during the credits), it won't appear in the menu before we encounter
|
|
the first packet. We cannot pre-parse the entire stream because it
|
|
would take hours (even without decoding it).
|
|
</para> </warning>
|
|
|
|
<para>
|
|
It is currently the responsibility of the input plugin to spawn the necessary
|
|
decoder threads. It must call <function> input_SelectES </function>
|
|
<parameter>( input_thread_t * p_input, es_descriptor_t * p_es )
|
|
</parameter> on the selected ES.
|
|
</para>
|
|
|
|
<para>
|
|
The stream descriptor also contains a list of areas. Areas are logical
|
|
discontinuities in the stream, for instance chapters and titles in a
|
|
DVD. There is only one area in TS and PS streams, though we could
|
|
use them when the PSM (or PAT/PMT) version changes. The goal is that
|
|
when you seek to another area, the input plugin loads the new stream
|
|
descriptor tree (otherwise the selected ID may be wrong).
|
|
</para>
|
|
|
|
</sect1>
|
|
|
|
<sect1> <title> Methods used by the interface </title>
|
|
|
|
<para>
|
|
Besides, <filename> input_ext-intf.c </filename>provides a few functions
|
|
to control the reading of the stream :
|
|
</para>
|
|
|
|
<itemizedlist>
|
|
<listitem> <para> <function> input_SetStatus </function>
|
|
<parameter> ( input_thread_t * p_input, int i_mode ) </parameter> :
|
|
Changes the pace of reading. <parameter> i_mode </parameter> can
|
|
be one of <constant> INPUT_STATUS_END, INPUT_STATUS_PLAY,
|
|
INPUT_STATUS_PAUSE, INPUT_STATUS_FASTER, INPUT_STATUS_SLOWER.
|
|
</constant> </para>
|
|
|
|
<note> <para> Internally, the pace of reading is determined
|
|
by the variable <parameter>
|
|
p_input->stream.control.i_rate</parameter>. The default
|
|
value is <constant> DEFAULT_RATE</constant>. The lower the
|
|
value, the faster the pace is. Rate changes are taken into account
|
|
in <function> input_ClockManageRef</function>. Pause is
|
|
accomplished by simply stopping the input thread (it is
|
|
then awaken by a pthread signal). In that case, decoders
|
|
will be stopped too. Please remember this if you do statistics
|
|
on decoding times (like <filename> src/video_parser/vpar_synchro.c
|
|
</filename> does). Don't call this function if <parameter>
|
|
p_input->b_pace_control </parameter> == 0.</para> </note>
|
|
</listitem>
|
|
|
|
<listitem> <para> <function> input_Seek </function> <parameter>
|
|
( input_thread_t * p_input, off_t i_position ) </parameter> :
|
|
Changes the offset of reading. Used to jump to another place in a
|
|
file. You <emphasis>mustn't</emphasis> call this function if
|
|
<parameter> p_input->stream.b_seekable </parameter> == 0.
|
|
The position is a number (usually long long, depends on your
|
|
libc) between <parameter>p_input->p_selected_area->i_start
|
|
</parameter> and <parameter>p_input->p_selected_area->i_size
|
|
</parameter> (current value is in <parameter>
|
|
p_input->p_selected_area->i_tell</parameter>). </para>
|
|
|
|
<note> <para> Multimedia files can be very large, especially
|
|
when we read a device like <filename> /dev/dvd</filename>, so
|
|
offsets must be 64 bits large. Under a lot of systems, like
|
|
FreeBSD, off_t are 64 bits by default, but it is not the
|
|
case under GNU libc 2.x. That is why we need to compile VLC
|
|
with -D_FILE_OFFSET_BITS=64 -D__USE_UNIX98. </para> </note>
|
|
|
|
<note> <title> Escaping stream discontinuities </title>
|
|
<para>
|
|
Changing the reading position at random can result in a
|
|
messed up stream, and the decoder which reads it may
|
|
segfault. To avoid this, we send several NULL packets
|
|
(ie. packets containing nothing but zeros) before changing
|
|
the reading position. Indeed, under most video and audio
|
|
formats, a long enough stream of zeros is an escape sequence
|
|
and the decoder can exit cleanly.
|
|
</para> </note>
|
|
</listitem>
|
|
|
|
<listitem> <para> <function> input_OffsetToTime </function>
|
|
<parameter> ( input_thread_t * p_input, char * psz_buffer,
|
|
off_t i_offset ) </parameter> : Converts an offset value to
|
|
a time coordinate (used for interface display).
|
|
[currently it is broken with MPEG-2 files]
|
|
</para> </listitem>
|
|
|
|
<listitem> <para> <function> input_ChangeES </function>
|
|
<parameter> ( input_thread_t * p_input, es_descriptor_t * p_es,
|
|
u8 i_cat ) </parameter> : Unselects all elementary streams of
|
|
type <parameter> i_cat </parameter> and selects <parameter>
|
|
p_es</parameter>. Used for instance to change language or
|
|
subtitle track.
|
|
</para> </listitem>
|
|
|
|
<listitem> <para> <function> input_ToggleES </function>
|
|
<parameter> ( input_thread_t * p_input, es_descriptor_t * p_es,
|
|
boolean_t b_select ) </parameter> : This is the clean way to
|
|
select or unselect a particular elementary stream from the
|
|
interface.
|
|
</para> </listitem>
|
|
</itemizedlist>
|
|
|
|
</sect1>
|
|
|
|
<sect1 id="input_buff"> <title> Buffers management </title>
|
|
|
|
<para>
|
|
Input plugins must implement a way to allocate and deallocate packets
|
|
(whose structures will be described in the next chapter). We
|
|
basically need four functions :
|
|
</para>
|
|
|
|
<itemizedlist>
|
|
<listitem> <para> <function> pf_new_packet </function>
|
|
<parameter> ( void * p_private_data, size_t i_buffer_size )
|
|
</parameter> :
|
|
Allocates a new <type> data_packet_t </type> and an associated
|
|
buffer of i_buffer_size bytes.
|
|
</para> </listitem>
|
|
|
|
<listitem> <para> <function> pf_new_pes </function>
|
|
<parameter> ( void * p_private_data ) </parameter> :
|
|
Allocates a new <type> pes_packet_t</type>.
|
|
</para> </listitem>
|
|
|
|
<listitem> <para> <function> pf_delete_packet </function>
|
|
<parameter> ( void * p_private_data, data_packet_t * p_data )
|
|
</parameter> :
|
|
Deallocates <parameter> p_data</parameter>.
|
|
</para> </listitem>
|
|
|
|
<listitem> <para> <function> pf_delete_pes </function>
|
|
<parameter> ( void * p_private_data, pes_packet_t * p_pes )
|
|
</parameter> :
|
|
Deallocates <parameter> p_pes</parameter>.
|
|
</para> </listitem>
|
|
</itemizedlist>
|
|
|
|
<para>
|
|
All functions are given <parameter> p_input->p_method_data </parameter>
|
|
as first parameter, so that you can keep records of allocated and freed
|
|
packets.
|
|
</para>
|
|
|
|
<note> <title> Buffers management strategies </title>
|
|
<para> Buffers management can be done in three ways : </para>
|
|
|
|
<orderedlist>
|
|
<listitem> <para> <emphasis> Traditional libc allocation </emphasis> :
|
|
For a long time we have used in the PS plugin
|
|
<function> malloc()
|
|
</function> and <function> free() </function> every time
|
|
we needed to allocate or deallocate a packet. Contrary
|
|
to a popular belief, it is not <emphasis>that</emphasis>
|
|
slow.
|
|
</para> </listitem>
|
|
|
|
<listitem> <para> <emphasis> Netlist </emphasis> :
|
|
In this method we allocate a very big buffer at the
|
|
beginning of the problem, and then manage a list of pointers
|
|
to free packets (the "netlist"). This only works well if
|
|
all packets have the same size. It is used for long for
|
|
the TS input. The DVD plugin also uses it, but adds a
|
|
<emphasis> refcount </emphasis> flag because buffers (2048
|
|
bytes) can be shared among several packets. It is now
|
|
deprecated and won't be documented.
|
|
</para> </listitem>
|
|
|
|
<listitem> <para> <emphasis> Buffer cache </emphasis> :
|
|
We are currently developing a new method. It is
|
|
already in use in the PS plugin. The idea is to call
|
|
<function> malloc() </function> and <function> free()
|
|
</function> to absorb stream irregularities, but re-use
|
|
all allocated buffers via a cache system. We are
|
|
extending it so that it can be used in any plugin without
|
|
performance hit, but it is currently left undocumented.
|
|
</para> </listitem>
|
|
</orderedlist>
|
|
</note>
|
|
</sect1>
|
|
|
|
<sect1> <title> Demultiplexing the stream </title>
|
|
|
|
<para>
|
|
After being read by <function> pf_read </function>, your plugin must
|
|
give a function pointer to the demultiplexer function. The demultiplexer
|
|
is responsible for parsing the packet, gathering PES, and feeding decoders.
|
|
</para>
|
|
|
|
<para>
|
|
Demultiplexers for standard MPEG structures (PS and TS) have already
|
|
been written. You just need to indicate <function> input_DemuxPS
|
|
</function> and <function> input_DemuxTS </function> for <function>
|
|
pf_demux</function>. You can also write your own demultiplexer.
|
|
</para>
|
|
|
|
<para>
|
|
It is not the purpose of this document to describe the different levels
|
|
of encapsulation in an MPEG stream. Please refer to your MPEG specification
|
|
for that.
|
|
</para>
|
|
|
|
</sect1>
|
|
|
|
</chapter>
|
|
|