Thursday, July 23, 2009

Automating Video Conversion, Part 1

Part of my role here at Calvary is “other duties as assigned”. This blog post today is the first in a series with the goal of “documentation for myself and my coworkers category”. This post is from the viewpoint of an IT guy trying to help coworkers out (when you really need a Broadcast engineer for all this video stuff). You’ve been warned ;)

A little background:

Recently we started using MediaShout for some ministries (Youth, Children’s, etc.) It’s a very nice product, but definitely has some quirks. One of those is getting it to play QuickTime files. The nuances of video formats and vendors makes my head hurt, so we’ll skip most of it.

Our Children’s Ministry recently started using some curriculum from Re:Think Group. They get a “packaged” solution, and can adapt it to their needs. Unfortunately, Rethink doesn’t offer their files in a format that works well (supposedly it works perfect if you have Mac’s). Unfortunately we discovered this after purchasing the curriculum (which is marketed as working on Windows OOB).

Note to curriculum vendors: my next post will be highlighting automating this process. You’ll earn big customer loyalty if you provide your video in a format that a non-video/non-techie can use.

So, as part of my helping, I was tasked with finding a solution. After spending a few hours working with the people at Rethink, we reached the conclusion that the only way to get their files to work reliably was to convert them (they primarily provided some h264 format that didn’t work for us). Rethink recommended QuickTime Pro ($30). Fortunately we had a license handy to give it a go.

Using Quicktime to convert video files

  1. Install QuickTime player, and then register your "Pro” version (the free version doesn’t convert files).
  2. Open your source file in QuickTime (if it plays w/ QuickTime, you should be able to convert it)
  3. File –> Export (specifying the destination directory)
    07-2009 Video Conversion Automation 01
  4. Specify your export settings. In my case I’m exporting with the intent of using these in MediaShout, and I’ve previously decided on “mp4” files:
    • Choose “Movie to MPEG-4” in the “Export” menu:
      07-2009 Video Conversion Automation 02
    • Click the “Options” button. This will bring up the Export Settings dialogue. We’ll be changing a lot of these settings:
      Note: many of these options stay in the format that you last used, i.e. you don’t have to change them every time.
      07-2009 Video Conversion Automation 03
    • First of all, change the “File Format” to “MP4” without the (ISMA). Don’t ask me why, I don’t know. What I do know is that vanilla MP4 worked, and MP4 (ISMA) didn’t work in our situation:
      07-2009 Video Conversion Automation 04
    • Next, we need to specify a video format. We decided on H.264, because it provides high quality, and high compression, and the source content is H.264:
      07-2009 Video Conversion Automation 05
    • Now we need to specify the “Image Size”. We’re just looking to do a format conversion (no scaling), so choose “current”:
      07-2009 Video Conversion Automation 06
    • Next we need to change the “Data Rate” (the amount of disk space used during compression). We’re going with 6000kbits/sec (that’s what I picked). Go ahead and replace the current setting with “6000”:
      07-2009 Video Conversion Automation 07
    • Now, we need to change some other, more advanced options. Click on the “Video Options” button:
      07-2009 Video Conversion Automation 08
    • We’re going to stick with the “Main” profile (that’s the default), but change the “encoding mode” to “Best Quality (Multi-Pass):
      07-2009 Video Conversion Automation 09
      Click OK
    • At this point, we’ve set all the options. Before moving on, let’s review the options we’ve set:
      • H.264 Video (in an mp4 “wrapper")
      • maintain the video size/frame size
      • use a video compression rate of 6000kbps (6mbps)
      • use the default of 30 fps
      • AAC-LC audio, Stereo
      • 128kbps audio compression rate
      • 44.1 kHz
        07-2009 Video Conversion Automation 10
    • Click OK, name your file, and click “Save”
      07-2009 Video Conversion Automation 11
    • Go get a cup of coffee, and then (hopefully) it will be done converting your file. If you want to, you can start up multiple conversions simultaneously.
      07-2009 Video Conversion Automation 12

Note: We decided to use Sorenson Squeeze to automate this (detailed in my next post)

Tuesday, July 7, 2009

Clonezilla, Imaging and Sysprep

My apologies for the long wait on a new posting. Chalk it up to living life ;) This is the first in a 2-part post.

This post here is of the "technical, document it so I don't forget" nature.

One of the "projects" that I've been working on lately is getting a more standardized installation of software on our computers.

To help accomplish this, I've been working with a volunteer, Phil, on using Clonezilla (currently the Live version). Up until now, we've been using all WindowsXP boxes. Here's an idea of the process as it happens (some of this is scripted):

NOTE: All installs are done as a local "admin" account, not as a domain account (more on that later). We use an internal software repository. This order is simply a guide, and what we've found works best from trial and error.

  • Install Windows XP on the machine (using the proper media)
  • Install any needed service packs
  • Join the machine to our AD domain
  • Add local administrator accounts, including setting passwords (we'll call these accounts "admin1" and "admin2" for now)
  • BIOS update (if applicable)
  • "System Software" ala Dell Desktop System Software
  • Chipset Software/Drivers
  • Graphics Card software/drivers
  • NIC software/drivers
  • If a laptop, management software, ala Dell Quickset
  • Audio card software/drivers
  • Wireless Card software
  • Touchpad/Pointing software
  • vPro/TPM software
  • Antivirus/Anti-Malware software (we use Sunbelt Vipre, a great product)
  • MS Office
  • MS Office addendum's (visio, etc.)
  • MS Office service pack's
  • Microsoft .NET Framework 3.5 & patches
  • Paint.NET
  • Foxit Reader
  • Mozilla Firefox
  • XP Previous Versions client
  • UltraVNC (used for in-house remote tech support)
  • Verify UltraVNC setup is working
  • Adobe Flash Player plugins
  • CDBurnerXP
  • ACS Facility Scheduler
  • ACS People Suite
  • ACS The Ministry Scheduler
  • Copy "Sysprep" folder to C:\ drive
  • Run Windows Update (preferably use Microsoft Update)
  • Setup and copy a clean user profile
    • This is why we have 2 local admins. You need "admin2" so that you can copy the profile info from "admin1" to the "default user" profile
  • Run Sysprep
    • we use "C:\sysprep\sysprep -reseal -quiet -mini -pnp"
    • This cleans the SID's from the system, does all it's work with little/no intervention, and also "resets" the system (similar to how a new PC comes from the OEM)
    • If you also create a "sysprep.inf" file, you can make the complete setup "unattended"
  • Image the machine. We use Clonezilla
  • Reboot, and watch the magic happen
  • Properly name the computer, and add it to the domain

Some possible "gotcha's":

  • If you use the "pnp" switch, upon first bootup (after you re-image a system), you will have to wait 3-6 minutes for Plug-n-Play to redetect hardware, but you get the added advantage of having images that can traverse hardware types (Core2Duo -> P4, etc)
  • I have ran into issues when working with large platform changes (i.e. very old P4 ServerWorks architecture to a new AMD architecture). This happens because the system doesn't have the proper IDE/ATA drivers in place, and sometimes different drivers don't play well together.
  • If you are moving between different hardware with one image, you need to include the proper drivers for all your systems before you image the machine.

This process works for us, even though it's very 2004. There are now better solutions, and Vista requires some rethinking/reworking of this process. At some point we'll upgrade. How do you handle imaging/deployment of your machine's?

Saturday, February 14, 2009

House Purchase

Today my wife Michelle and I started the process of purchasing a house.

Details at our family blog.

Tuesday, December 23, 2008

ESXi monitoring, for free!

A few months ago I transitioned us from VMware Server to VMware ESXi, booting off of a USB flash drive. If you don't know about server virtualization, VMware ESXi is a great way to get your feet wet, and it's a stable, production-ready (IMO) product.

However, one of the things that eluded me (in both the "Server" flavor and the "ESXi" flavor) was proper monitoring. Sure, I could setup data on each guest VM, but that didn't give me any info on the host.

Fast forward to yesterday, and I hear through the grapevine that Veeam is offering a free ESXi monitoring tool. Go get it here.

I'm just downloading it today, but if it does what the "Features and Benefits" page says, then this will be a new must-have in my toolkit. More updates to come (hopefully) as I try it out.

Sunday, December 21, 2008

HELP: ACS TMS to Facility Scheduler Conversion

One of my current projects at Calvary is to work on moving us to the latest release of the ACS People Suite (10.1.1.2). Part of this process is getting all of our ACS The Ministry Scheduler data into ACS Facility Scheduler.

ACS Facility Scheduler is an "on demand" product. This means that all the actual data sits on ACS' servers, and they handle data integrity, backup, etc. for you. Months ago, we looked at converting to Facility Scheduler before ACS 10.0 came out. At the time, there were some issues we had (features missing). So, we waited until those features came out. When they arrived, I had other projects taking precedence, and consequently we rolled it all into the 10.x upgrade.

As part of our upgrade process, I found out that ACS has a great conversion tool to transfer your current ACS TMS data into Facility Scheduler. I first used this tool when we were testing the feature set. Before this 10.x upgrade, I got in touch with one of the ACS people about "resetting" our data so I could re-upload the current data. He kindly let me know that the latest version of the tool had this functionality built in!

However, if you take a look at the ACS Knowledge Base article or Facility Scheduler FAQ on the subject, you find that you can no longer download the conversion tool (and it doesn't show up in the previous "client downloads" section either).

 

Does anyone out there have the file "tmsconversion.exe" or "ACS_TMS_to_FS_Conversion.exe", the converter to move from The Ministry Scheduler to Facility Scheduler? If so, please shoot me an email: sross *at* calvaryonline.cc

Friday, October 3, 2008

Moving an Ubuntu virtual machine from VMware Server to ESXi (on a PE1950)

Wednesday I migrated my PE1950 from VMware Server (1.0.2!) to ESXi 3.5 Update 2. During the process I ran into some issues moving my Ubuntu 6.06 LTS VM to ESXi. Here's the play-by-play (including my hardware upgrade).

  1. Copy the VM's off of the VMware Server.
  2. Verify the copied VM's work ok, and that you have valid backups.
  3. Shutdown the PE1950.
  4. Update the BIOS on the PE1950. Without a BIOS update, ESXi will not run correctly.
    - Can you believe I was running 1.x, when we're now at 2.3.x! This box has been very, very reliable.
  5. Unrack the PE1950, and replace the SAS 5/iR (no RAID) controller with a PERC 6/i controller.
    - ESXi needs a hardware RAID controller.
    - I was previously running software RAID-1 on the Ubuntu LTS host. We needed a reliable system, since this box had become mission-critical.
  6. Install ESXi onto a USB flash drive (>=1GB).
  7. Boot the PE1950, and setup the RAID array (2x300GB 7200RPM SATA in RAID-1).
  8. After the array has initialized, reboot with the USB Flash drive plugged in (preferably to one of the rear USB ports).
  9. Enter the BIOS (F2), and modify the boot order.
    - I set the USB Flash Drive's mode to "Hard disk"
    - Modify the boot order to include the USB flash drive as taking higher priority than the PERC array.
    - Save and exit the BIOS.
  10. Setup ESXi.
    - ESXi will give you the IP you need for setting up the Virtual Infrastructure client, etc.
    - Your RAID-1 array will be setup as your primary datastore (datastore1).
  11. Use VMware Converter to move the vm's to the new ESXi box.
  12. Boot up the Ubuntu guest OS.

 

Upon boot, you'll notice that the Ubuntu machine has no network connectivity. Here's how you fix it (commands you need to type are in bold):

  1. Install VMware tools on the guest os (it's probably outdated)
    • In the VMware Infrastructure Client, choose the VM, and then go to Inventory->Virtual Machine->Install/Upgrade VMware Tools
    • log into the ubuntu console
    • elevate your privileges to root level by running sudo su
    • mount the cd-rom drive: mount /media/cdrom0
    • change directories to the cdrom drive: cd /media/cdrom0
    • copy the vmware tools tar archive to your tmp directory (making sure you pay attention to the name of your archive, including case):
      cp VMwareTools-3.5.0-110271.tar.gz /tmp/
    • change to the tmp directory: cd /tmp
    • extract the tar file: tar -xvf VMwareTools-3.5.0-110271.tar.gz
    • change directories to the vmware-tools installer: cd vmware-tools-distrib
    • run the vmware tools installer script: ./vmware-install.pl
  2. Restart your networking: /etc/init.d/networking restart
  3. Check to see if your NIC is now working properly. You can check your interfaces using the following command: ifconfig -a
    If you are receiving an IP properly, you're probably OK. This didn't work for me.
    DO NOT complete the following steps unless you have no network connectivity
  4. Shut down the VM: shutdown -h now (remember, we elevated our privileges earlier to root)
  5. Remove any NIC's that are currently in the VM.
  6. After removing any NIC's that are currently in the VM, add a new NIC.
  7. Boot the VM
  8. I now had a NIC that my system recognized, but I wasn't getting an IP. The issue was with my interfaces file.
    • Contents of /etc/network/interfaces:
      # The loopback network interface
      auto lo
      iface lo inet loopback

      # The primary network interface
      auto eth0
      iface eth0 inet dhcp
    • Notice how it lists "eth0" When I ran "ifconfig -a" earlier, I received eth1 as an interface, not eth0
  9. Change eth0 to eth1 in my interfaces file: vi /etc/network/interfaces (replacing eth0 with eth1)
  10. Restart networking: /etc/init.d/networking restart

At this point, everything was working well.

Thursday, October 2, 2008

VMware ESXi (bootable) USB flash creation tip

Yesterday I went to install VMware ESXi on a Poweredge 1950. All along I wanted to get the system setup with a USB flash drive (and not use the onboard storage as my boot disk).

I did some research, and this blog post seemed to be the most complete posting on creating your own ESXi bootable flash drive.

So, I downloaded the ESXi installable ISO, opened up 7-Zip, and went for it.
I was very surprised that every time I tried to image the flash drive, I got an error in WinImage. Now, this was running on my Vista x64 box, so I went ahead and fired up a VM w/ XP Pro 32-bit. At that point, I attempted to re-image the USB flash drive, and things worked as planned

Moral of the story: Don't try and create a bootable USB flash drive using Winimage on Vista x64, it won't work! Use VMware (or another computer) to create the flash drive's ESXi install (apparently on a 32-bit OS).