Today we released 3.1.0, containing a number of significant new features along with a few bug fixes. The features are:
- Improved quality of web-optimized videos
- Images generated up to 3 times faster
- Auto-rotation of images and videos
- Synchronization up to 300% faster and uses less server memory
- Easier Active Directory setup and integration
- Improved performance for large numbers of user accounts
- On-demand rotation of videos
- Support for 3GP videos
This release is a free upgrade to 3.0 license holders. Upgrading is easy – just copy the files from the upgrade package over your existing web application. Installations and upgrades for other scenarios are in the Administrator’s Guide.
Video Improvements
When the Gallery Server Pro Binary Pack is installed, web-optimized versions of video files are created when you upload a file or the synchronization determines that an optimized video is needed. GSP 3.0.X created H.264 MP4 files that were highly compressed, resulting in small files that played quickly in all modern browsers.
However, I have come to believe they are *too* optimized and the loss in video quality was too much of a tradeoff. Version 3.1 changes this by creating videos with a higher quality, albeit with slightly higher file sizes as a result.
Specifically, what we changed in 3.1 was the FFmpeg arguments for the All video => .MP4 conversion process. You can manage these settings on the Video & Audio page in the site admin area.
In 3.0.X, the FFmpeg arguments were this:
-y -i "{SourceFilePath}" -vf "scale=min(iw*min(640/iw\,480/ih)\,iw):min(ih*min(640/iw\,480/ih)\,ih)" -b:v 384k -vcodec libx264 -flags +loop+mv4 -cmp 256 -partitions +parti4x4+parti8x8+partp4x4+partp8x8 -subq 6 -trellis 0 -refs 5 -bf 0 -coder 0 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -qmin 10 -qmax 51 -qdiff 4 -ac 1 -ar 16000 -r 13 -ab 32000 -movflags +faststart "{DestinationFilePath}"
In 3.1, it is now this:
-y -i "{SourceFilePath}" -vf "scale=min(iw*min(640/iw\,480/ih)\,iw):min(ih*min(640/iw\,480/ih)\,ih){AutoRotateFilter}" -vcodec libx264 -movflags +faststart -metadata:s:v:0 rotate=0 "{DestinationFilePath}"
Notice that a lot of the optimization settings have been removed, allowing FFmpeg to use default settings in many areas. There are also two other changes:
- New {AutoRotateFilter} replacement parameter—This string is replaced with a rotation filter that automatically rotates the video to the correct orientation based on the orientation flag that may be present in the original video file. This feature requires a recent version of FFmpeg so if you haven’t updated the binary pack lately, be sure to do so.
- Removal of orientation flag—The string “-metadata:s:v:0 rotate=0” tells FFmpeg not to include the orientation flag in the web-optimized video. Without this, the file would have the same rotation flag as the original file even though it is now rotated.
To generate a new set of web-optimized videos based on the new setting, run a synchronization with the option Rebuild optimized versions selected.
If you want to continue generating the highly compressed videos like it was done in 3.0.X, change the FFmpeg arguments for the All video => .MP4 setting back to its original value, except with the new settings included, like this:
-y -i "{SourceFilePath}" -vf "scale=min(iw*min(640/iw\,480/ih)\,iw):min(ih*min(640/iw\,480/ih)\,ih){AutoRotateFilter}" -b:v 384k -vcodec libx264 -flags +loop+mv4 -cmp 256 -partitions +parti4x4+parti8x8+partp4x4+partp8x8 -subq 6 -trellis 0 -refs 5 -bf 0 -coder 0 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -qmin 10 -qmax 51 -qdiff 4 -ac 1 -ar 16000 -r 13 -ab 32000 -movflags +faststart -metadata:s:v:0 rotate=0 "{DestinationFilePath}"
Up to 3X faster image generation
Versions of GSP prior to 3.1 used the GDI+ library in the .NET Framework to generate the thumbnail and optimized images for most image types. (TIFF being one exception where ImageMagick is used instead.) This has worked reliably but the algorithms are pretty slow.
Starting with this release, GSP now uses the WPF classes in the .NET Framework. They are much faster and provide the same or better quality as the old technique.
The WPF classes require a higher level of permissions than the old GDI+ classes. The code automatically detects the current trust level of the application. When the app is running in full trust, the WPF classes are used. Anything less and it falls back to the GDI+ classes. Thus those users who are running in full trust get the benefits while we still support those of you running in reduced trust scenarios.
Auto-rotation of images and video
A frequent request has been the detection of orientation in photos and videos and then auto-rotating them as need to make them right side up. It’s been discussed in the forums here and here and on the internet here. Initially I was hesitant to add this feature due to the terribly inconsistent way it’s been implemented in applications. My biggest worry was adding a feature that ended up rotating files that shouldn’t be rotated, essentially corrupting the media library. However, I got a good reminder of how useful this feature would be when I started uploading images and videos from my Samsung Galaxy S4 and discovered I had to tilt my head to see half the items.
As a result, we spent some time deliberating how best to implement this feature, and I think we’ve come up with a robust solution. For starters, when you upload or synchronize a photo or video that has orientation metadata embedded in it, the gallery extracts that information and stores it in the metadata table. You can see it in the new orientation meta item:
Second, when the gallery creates the thumbnail and optimized files, it uses the orientation flag to determine if a rotated version of the original should be created. The original file is not rotated or modified in any way during this process.
In the above example, the original photo is rotated 90 degrees counter clockwise, as seen here in Paint.NET:
But when we add it to the gallery, GSP detects the orientation flag in the metadata and uses it to create the thumbnail and optimized images rotated 90 degrees clockwise. Again, the original file is unchanged, which you can see if you use the Download/Share button to download the original image from the gallery.
But what happens if you rotate a photo or video from within GSP? This is a case where the original file *is* modified. For example, if we go to the rotate page for the above photo, leave the default selected ‘UP’ icon at the top of the image, and then click rotate, guess what happens?
Since the original image is rotated 90 degrees CCW, GSP will rotate it 90 degrees CW so that it is oriented the same way as the thumbnail and optimized image. Also, to prevent issues in the future, the orientation flag in the original file is removed. If we left it in, we would experience potentially confusing behavior in other apps and in GSP, since we’d have a right side up photo whose metadata indicates it’s rotated to the left.
After the rotation, the image still appears right side up. But notice the orientation meta item is no longer present:
If we open the original file in Paint.NET, we see it is rotated 90 degrees clockwise from its original orientation:
We used a photo as our example here, but videos behave the same way. When the binary pack is installed, GSP uses FFmpeg to detect the orientation and then creates the thumbnail and optimized files with the required amount of rotation. Earlier in this post we looked at the new {AutoRotateFilter} parameter in the FFmpeg arguments setting. That is key to this feature. The original video is unmodified until you manually invoke a rotation, in which case the original is recreated with the desired rotation and any embedded orientation flag is removed.
What if you don’t want auto-rotation? For videos, all you need to do is remove the {AutoRotateFilter} parameter from the encoder settings. For images, the behavior is hard-coded and cannot be turned off. I considered adding a setting to disable the feature, but decided to wait to see if it’s really necessary. If you are affected by this and don’t want the auto-rotation feature, contact me and describe your situation.
Applying auto-rotate to your existing media objects
If you upgraded to 3.1 and want your existing media objects to be auto-rotated, follow a two-step process:
1. On the Metadata page, click the rebuild button next to the orientation meta item:
2. Run a synchronize with the ‘Overwrite thumbnail images’ and ‘Overwrite optimized versions’ options selected.
You may want to do a test in one album before recursively regenerating items for all your objects. If any existing images or videos have actual orientations that conflict with the orientation meta item, you will end up with files created with the wrong rotation. If you discover this to be the case, I recommend cleaning up your original files by rotating them with a 3rd party app or using a tool to remove the orientation meta item.
Synchronization up to 300% faster and uses less server memory
The synchronization algorithm was modified to be faster and use less server memory. The primary change that caused it be faster is the switch to using WPF to create thumbnail and optimized images as we described earlier, but the other important change is that the synchronization no longer loads all media objects into memory at the beginning of a sync. Instead, media objects are loaded on an as-needed basis as each album is processed.
In a test with photos that averaged 18 MB each, the synchronization process finished 300% faster in 3.1.0 than in 3.0.3! That was the best case scenario, as the performance benefits are greatest for large images. In another test where the images averaged 2.9 MB, the speed increase was a still respectable 43%.
SQL CE users will notice a smaller performance benefit since much of a synchronization is spent on database access, but the speed increase should still be noticeable.
Some web hosting providers automatically recycle the application pool when certain memory limits are reached. In some cases the limits are—in my opinion—ridiculously low (ahem...Arvixe Personal Class has a 250 MB limit).
These changes to the sync process should help reduce app recycling due to hitting memory limits. However, be aware that the sync process is inherently a memory-intensive operation and if you have trouble running the gallery on a cheap plan, you may need to upgrade.
Removed feature: Support for preserving media object record when original file has been moved to another directory
To support the performance and memory improvements of the synchronization process, one feature that has existed since the beginning of time had to be removed. Prior to 3.1, if you used Windows Explorer or some other tool to move an original media file from one directory to another, the synchronization process was able to detect the move and update the original media object record to indicate its new album location. It was able to do this by creating a hash key of each file and storing it in the HashKey column of the MediaObject table.
However, this technique had two problems. One, it required loading the hash keys of all media objects in memory at the start of each sync. For large galleries, this came at a performance cost and sometimes triggered an app recycle even before the sync could get underway. Two, the hash algorithm was imperfect because it was based on the file name and its creation timestamp, two values that might be duplicated in other media files. This caused an issue during synchronization for some users.
Because of these two issues, this feature was removed in 3.1.0, along with the HashKey column in the MediaObject table. Now if you manually move a file to another directory, the synchronization process will delete the old record in the MediaObject table and then create a new one. Any titles, captions, tags, or other data associated with the original record is lost.
Easier Active Directory setup and integration
One of the pain points in GSP has been the hoops one must jump through to integrate the gallery’s membership with Active Directory. With 3.1.0, the process has been greatly simplified. Now you no longer have to trick the gallery into running under .NET 2.0 just so you can use IIS Manager’s .NET Users applet to set up the admin account. Instead, you choose one of your AD accounts and specify the username and password in the install.txt file in the App_Data directory:
During installation, GSP detects the account info and automatically configures it as an administrator in the gallery.
You also no longer have to specify an AD account in web.config with update permission to Active Directory. Instead, the gallery need only be running under an identity with read permission to AD.
More information about AD integration is in the Administrator’s Guide.
Improved performance for large numbers of user accounts
The Manage Users page has been modified to perform better when there are large numbers of user accounts. This is especially helpful for Active Directory integrations where there are often tens of thousands of users. Notice a new textbox at the top that lets you search for a user:
The textbox has auto-complete functionality so when you type a few characters, a list of matching accounts appears.
When the gallery contains more than 1,000 accounts, the Manage Users page no longer renders all those accounts to the screen. Now they are hidden and you use the find user textbox to find the account you want to manage:
Support for 3GP videos
This was an easy feature to add but one that should help anyone with 3GP videos. These are treated the same as MP4 videos by the gallery, meaning that they are rendered in an HTML5 video element in browsers that support it and with Flash in the rest of the browsers.
