IBM Content Manager client server Video streaming

Trying to find out the size of the data block used to store, retrieve and transmit video/audio data (stored as BLOB) from the Content Manager Serevr to the client.


Thank you.
cyber-33Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Kent OlsenDBACommented:
Hi Cyber,

There are quite a few items in that process that can be called blocks.  Which one are you looking for?


Kent
cyber-33Author Commented:
Hi Kent,

I am trying to understand a concept behind the storage, retrieval and transmission of large binary objects that represent audio and video data. I am sure this data is broken up into smaller chunks (I believe Oracle refers to them as blocks). The blocks are sequential, they can be retrieved and transmitted independently.

I gave 500 points for this question because it requires in-depth understanding of the internals, although the question is trivial. The knowledge deserves a reward :)

Thank you!
Kent OlsenDBACommented:
Hi Cyber,

BLOB data in DB2 works differently.  As far as I know, the application will have to read the BLOCK object from the database and use it.

That said, I don't know of anything that would prevent a C/C++/Java application from doing its own blocking -- reading the data in chunks and using the chunks.  The application could also, theoretically, act as a streaming interface and simply stream the data to a file/device as it is being streamed to the application.

But a BLOB object is treated as a single entity within DB2.


Kent
Build an E-Commerce Site with Angular 5

Learn how to build an E-Commerce site with Angular 5, a JavaScript framework used by developers to build web, desktop, and mobile applications.

cyber-33Author Commented:
Kent,

I agree that a BLOB is a single entity. I am trying to figure out the organization of "sub-entities".

For example, when the BLOBs are transmitted over the network, they are transmitted in a sequence of network packets. Similarly, when they are retrieved from the hard drive, they are also NOT retrieved in 2GB chunks. Instead they BLOBs must be retrieved in some smaller data unit that are used by IBM Content Manager to store, retrieve and transmit the data.

Thank you for your help.
Kent OlsenDBACommented:

Hi Cyber,

The network layer is completely independent.  As far as TCP/IP is concerned, it was handed some data which is broken into chunks small enough to be transmitted on the network.  That happens in the IP stack well below the application.

The BLOB data is stored in DB2 as a BLOB data object (up to 2GB).  There is no built-in mechanism for accessing part of the BLOB item.

This page shows the table where Content Manager stores the BLOB objects.  It's a single column in a table.

 http://publib.boulder.ibm.com/infocenter/cmgmt/v8r3m0/index.jsp?topic=%2Fcom.ibm.apr.doc%2Frmblobs.htm


Kent
cyber-33Author Commented:
Kent,

The link you provided points to the table of a Resourse Manager (not the Content Manager). Also, I believe, BLOB data is stored in DB tables only if the data is sufficiently small. Usually the DB tables store only the pointers, while the BLOB data resides elsewhere.

Here is a good read on point:
http://publib.boulder.ibm.com/infocenter/db2luw/v9r7/index.jsp?topic=/com.ibm.db2.luw.admin.dbobj.doc/doc/c0054525.html

Thank you.
Kent OlsenDBACommented:
Sorry for the link.  Misread it.

BLOB/LOB data is stored within a DB2 database.  One of the DB2 restrictions is that row based data must fit within a logical page (block).  If the database uses 4K pages (blocks) then the maximum size of a row is slightly less than 4K to allow for the DB2 block/row headers.  The description in the link that you posted (nice link, by the way) states that an "inline" LOB can be up to 32K (less headers) in length.  The restriction is that it must fit into a page (block).

BLOB/LOB data is normally stored in a BLOB/LOB table and MUST be stored there if it will exceed 32K (or the page size).  BLOB/LOB tables are different than normal tables in that they aren't governed by the logical page constraint of normal tables.

There's a pretty good description here, and an implication about splitting BLOB/LOB objects for storage, but I don't think that that is what you're looking for.

  http://publib.boulder.ibm.com/infocenter/db2luw/v9r7/topic/com.ibm.db2.luw.admin.dbobj.doc/doc/c0052385.html



Kent

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
cyber-33Author Commented:
Kent,

Thank you for your help. I found the answer to my question on Page 75 of the pdf below:
http://www.redbooks.ibm.com/redbooks/pdfs/sg246410.pdf

IBM Content Manager uses Multimedia Filesystem that separate large media files into blocks. IBM Content Manager also uses "Wide Striping" - a feature that allows access of successive blocks of data from multiple disks and nodes.

For the benefit of the communiy I am pasting the content of the document above.

Isochronous file access
The Multimedia Filesystem uses real-time features of the AIX operating system
to prevent the kernel and other programs from interfering with isochronous data
delivery. It also implements real-time disk scheduling to execute disk I/O
operations in the proper order for achieving an uninterrupted flow of data to
clients.
Scalability
The Multimedia Filesystem provides a high degree of scalability, both in the
amount of data it can store and the data throughput it supports. The high
scalability is achieved by using large disk blocks and wide striping.
 Large disk blocks: To efficiently support multimedia and supercomputing, a
file system must maximize throughput from the available disks. Since disk
throughput is strongly related to disk block size, the Multimedia Filesystem
uses a large disk block size, 256 KB being the default. Conventional file
systems optimize to reduce space rather than to increase throughput, by
using small disk blocks; for example, the AIX native file system uses 4 KB
blocks. Since the Multimedia Filesystem can store several small files (or
partial blocks at the end of large files) together in a single large block, its
disk-space utilization is comparable to that of a conventional, small-block file
system.
 Wide striping: In both supercomputing and multimedia, much or all of the file
system activity is often directed at a single file. For example, many or all
clients of a Video-on-Demand system may be viewing the same movie
simultaneously the first time it becomes available. It is easy to image a
demand which is higher than the throughput of an individual disk (5-10 MB/s)
or even of an individual storage node in the SP2 (40 MB/s). Achieving higher
throughput than that of a disk or node from a single file requires striping it
across multiple disks and storage nodes, so that successive blocks of the file
go on different disks and nodes. Each Multimedia Filesystem can be striped
across as many as 512 disks. Since each file is striped evenly across all disks
in a stripe group, the disk load is inherently balanced, regardless of
file-access skew (the unevenness of demand for different files).
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
DB2

From novice to tech pro — start learning today.