important hint for P2 work

[filmus]

P2 kind of "mixes" video und audio data one after each other in short "pieces" on the hdd-tracks so that playing such material runs fast as audio is in the same hdd-track as the relating video. Reorganizing(!) a hdd with P2 material ends up in complete separating video and audio into different tracks, so that playing afterwards is much slower as the hdd heads have to jump permanently between video and corresponding audio on different tracks to play both together. Thus copying P2 material onto hdd from P2-cards using windows explorer destroys the P2 file-organization and "reorganizes" automatically - therefor it is recommended to transfer P2 material from P2 cards onto hdd using the Panasonic P2-viewer software (P2 card - to - virtual card on hdd), this keeps the video-audio clip organization on the hdd.

[THoff]

If I follow you correctly you are saying that the files for the audio and video are interleaved when P2 Viewer copies them, and that they are not interleaved when the files are copied by other methods. Your hypothesis is that P2 Viewer alternatingly copies small chunks from the audio and video files.

If you are correct and your system is unable to cope with non-interleaved files, you should disable the automatic disk defragmentation that is enabled in Windows Vista and Windows 7 by default, otherwise the file layout you prefer will be destroyed.

Having said that, a modern hard drive with a SATA/eSATA or better connection should have no trouble coping with non-interleaved files, DVCPro HD video is 100Mbps, and the audio is just 512Kbps. Since random read performance is worse than sequential read performance for mechanical drives, the only way I can see throughput becoming an issue is if your RT buffer setting is too low in Edius. How much memory is in your system and how much are you setting aside for buffers? Does the buffer counter climb when you start playing a sequence?

[filmus]

it is not a hypothesis but a fact (I am a filmmaking physicist) that P2 files are interleaved video/sound-wise. Destroying the P2 file interleaving structure by reorganizing a hdd leads to playback slowing down especially at LONG clips. We recognized and measured it. Our "remedy" is to put all our P2 data onto disks (eSATA) that are never reorganized. One can easily see that effect when putting P2 data onto an empty disk and then check the fragmentation of this disk: it says "hdd needs defragmentation" - if you do that, you will loose performance concerning P2 playback. This concerns not only DCVCPRO HD but also the AVCintra codecs. One should aim to keep the P2 data organization structure as on the P2 cards. It is just a tip - nothing more...we found out when looking for the reason why P2 material sometimes stutters and sometimes not at the same hardware configuration.

[tingsern]

Hmmm. I have been using P2 data for years on my workstation (using SCSI RAID) and have been re-organising data on my harddrives using PerfectDisk very frequently. Sometimes 2 times a day. I have never suffered from any I/O problems that you report.

[filmus]

good for you - the problem rises only at hardware configurations that have critical data-speed bottlenecks or under complex timeline contents when the pc's data transfer rates are at the ceiling. It is just a hint for a reason if P2 playback suddenly stutters - then one might reorganize his hdd as a remedy but acts completely contraproductive...

[Paranova Films]

we are using P2 files from a lot, in fact, a P2 was our first HD camera, and we really didn't have mayhor problems transfering or managing data... as you said: good for us! but i think this is a very interesting hint, and, (who knows) maybe it will get a better performance...

[THoff]

It is absolutely true that P2 data is interleaved when you record it due to the fact that the audio and video is written to disk as it becomes available during the recording.

What happens when you transfer it to a computer is an entirely different story. Panasonic promotes the P2 workflow as being "IT based", meaning that you just copy data rather than following special ingesting or capturing methods. Dragging files off the cards or using commandline copy commands, the interleaved organization of the data will be destroyed.

I'll run some tests when I get home to see if P2 Viewer is indeed cognizant of this and alternatingly copies chunks from the audio and video files. It is more work and less efficient to do it this way, so I'd be surprised if the Panasonic programmers went out of their way to do this. Furthermore, the NTFS and FAT32 clustering will undo some of the interleaving.

[THoff]

I can now assure you that Panasonic's P2 Viewer (at least with V3.6.18.0, the latest publicly available release) does NOT interleave the audio and video data when it copies it to your hard drive. Any benefit you may see from not defragmenting your hard drive is coincidental.

I've attached a screen shot of an activity trace using SysInternal's Process Monitor, as well as an Excel spreadsheet with the formatted logs of the entire ingest of a P2 card.

P2 Viewer first ingests all the audio files, then all the video files.

[filmus]

you are using P2 viewer's ingestion routine - but I am talking about copying P2 material within the P2 viewer (after switching on "show the secondary bin") from the P2 card to a VIRTUAL P2 card by dragging the clips from the P2 card window to the P2 virtual card windows. Try it and see the difference...

[THoff]

I did what you suggested, it made no difference. Even when creating a virtual P2 card and then dragging files between the physical P2 card and the virtual one in P2 Viewer, the audio and video files are copied separately and in their entirety without interleaving their data.

As before, a screenshot and a spreadsheet with a log of the operations is attached.

[tingsern]

I have been using P2 since 2001. I have never seen such behavior so far on my PC. I think if you have adequate hardware (meaning I/O subsystem) and a fast processor, you are most unlikely to see such performance problem.