Tuesday, November 30, 2010

What's in a number? Calculating the size of your image library

Beginning with Mac OS 10.6 Apple's Finder now calculates the size of your drive (internal and external) a bit differently than it did before. The idea now, apparently, is to provide the user with a number that appears "more accurate" than it might have before, when in fact the size reported in previous versions of Mac OS X simply allowed for a cushion.

For example, on a 2-tb external drive, my older desktop (OS 10.5.8) reports the space remaining as 1.07 while my laptop (with 10.6.5) reports 1.18 remaining. But, both machines report the same number of bytes, however (790 billion and change).  (By the way, a Windows machine reports the drive size the same way as the older Mac OS X.)

While I can't reproduce the complex explanations for how drive sizes are calculated (nor would I want to), suffice it to say that when you see your drive listed in Snow Leopard it attempts to provide you with a reading that is more in line with the accurate byte calculation. That is, 790 gigs used (to represent the 790+ billion bytes).

Now, aren't you glad you asked?

Wednesday, November 24, 2010

Previewing problems in Bridge CS4?

If you're having problems previewing images in folders you've previously indexed in Adobe Bridge CS4 try this:


1. In Bridge go to Edit>Find (Command+F)
2. Select the folder where the problem seems to occur most often (alternatively select all your image folders)
3. In the search criteria drop-downs select "keywords" and then type in the broadest keyword in your library, one that might cover all your images (such as your name, if you tag all your images that way  for example)
4. Make sure to check BOTH Include all subfolders and Include Non-indexed files
4. Click find and then walk away. Depending on how large your image library is, this might take a while (or overnight).


We've have found that using this method is one way to "clean up" any re-indexing issues that might arise in Bridge's caching process after months of re-indexing folders.


There are, of course, no guarantees so be sure to keep any eye on your caching numbers in the Library.

Friday, November 19, 2010

Adobe Bridge CS4 Previewing problem

It's been over a year since our department started using a central image library and, with only 2 or 3  minor issues, the print and web teams have experienced few problems accessing the photos using Adobe Bridge CS4. All machines have undergone both initial and repeated reindexing have produced no unexpected anomalies. Recently, however, several machines have begun to experience problems in loading previews of several recently uploaded folders.

Remember, our image library consists of a series of folders (by date), each with a varying number of images inside (but rarely over 2k or more images). Typically, once a folder of photos is cached (or "indexed) in Bridge, the program creates a separate subfolder for the thumbnails and another subfolder for the previews, all within Bridge's cache folders. (Located in the user's Library on a Mac.)

Initial indexing/caching creates all the thumbnails and previews for that one folder. Subsequent "recaching/re-indexing" only adds any additional metadata that might have been modified since the initial caching -- new images are almost never added to folders within the library.

Now, to the issue.

Beginning several weeks ago -- the timing is, unfortunately, uncertain - I began experiencing a preview loading problem on both the benchmark machines. (Two Macs, one desktop one laptop that serve as the most accurate and up-to-date sources for accessing the library.) The problem was first noticed in the search results window: previews appeared pixelated in the preview window and often remained that way. In addition, when attempting to review (or use the slideshow) feature, the images would often remain pixelated for some seconds or not load at all.

At first I thought this was isolated and temporary. But I then learned that several others were experiencing the same problem. (Only Macs, though. The two Windows machines, not overly used but one kept up-to-date, seemed fine.)

Modifying the preview generating option in the Path Bar made a short-term difference on a couple of machines, but the option was already selected on a couple of other machines exhibiting the loading problem.

When I began investigating the caching behavior of the problem machines, it soon became clear that the problem was deeper. By examining the cache folders inside of the Library cache folder structure, I could see that previews were not being generated even when the folder(s) were reindexed. In fact, I even tried purging and rebuilding the cache for several folders and watched the preview cache folder. I could easily see that not all the previews were being built and indexed.

The question now is why is this happening? The corollary is why is this happening on just a few machines? All the machines, it should be noted, are of the same vintage, and of robust features, and sharing the same OS and software.

Tuesday, October 19, 2010

Odd error messages when modifying metadata

It's unknown what these messages mean exactly, other than they cause havoc if you're attempting to tag a large number of images at a time. You'll have to force quit Bridge since there's no other way to stop these messages from popping up without clicking on them one at a time. 


No pattern has been found, aside from perhaps they seem to be more prevalent with RAW files but that's not an absolute.


Any ideas?


Tuesday, October 12, 2010

Keywording tips

There is a great deal of chatter about keywords and how they can make archiving photos (and other digital assets) accessible. The simple fact is they can indeed make life infinitely more enjoyable for the digital archivist and searchable for the designer or writer. But -- and this is a BIG but - you have to follow a few simple rules.

Identify and clarify an initial keyword list. This is best done in conjunction with your team's designers and writers. Once identified, the list can be built upon and developed as needed.

Deciding is your keyword list for internal or external use.

Know your image environment. If you're working primarily in one field use the words most associated with that field. For example, if you're in higher education, chances are slim you will need to search using military-oriented keywords. "Academics" or "History" or "Student," yes, but probably not ""armor," "platoon," or "squad."

Keep the keyword list short, sweet and relevant. There are a few exceptions to this rule, however. For example, we have a number of "historical" images representing old academic programs no longer taught at JWU: "keypunch" and "office machines" to name just two. I've tagged those programs using relevant keywords although I am almost certain those search terms will never be used again.

Also, if I'm faced with tagging an image that doesn't reflect any of the previously identified keywords, add a new keyword. You can always go back later and modify it -- but if you don't identify the keyword up front you've lost the opportunity to tag that image or group of images accurately.

For more in-depth discussion of this often misunderstood and misapplied concept, visit Peter Krough's illuminating article in DPBestFlow online.

Friday, October 1, 2010

Adding Metadata - bugs and glitches

An ongoing problem -- one that has frustrated me from the very beginning of this project -- is the error message that continually crops up whenever I try to add, delete, or otherwise modify the metadata in an image or group of images in the library. (I began this discussion this in my previous post.)

Specifically, the problem usually occurs in the "Search results" window (but not always). What happens is, when I try and modify the metadata of an image or a group of images and will often (but not always) get an error message, saying that such-and-such image cannot be modified, no reason given. Furthermore, if you have selected a number of images to be modified, you have to manually click through each one. More frustration.

Much of the time the problem can be "worked around" by going to the specific folder where the images are located and modifying them there, rather than in the search results window. Clunky but it does work. Sometimes the problem arises when dealing with Camera RAW files (NEF has been a primary culprit here). as I noted in my previous post, one can simply convert RAW files to DNG and the problem goes away. Plus it lowers the overall file size and no sidecar XMP files to deal with.

If anyone has any ideas or thoughts about this, I'd love to hear them. And share them.

Wednesday, September 29, 2010

Converting RAW to DNG - an absolute necessity now

Back in April I posted a discussion of the rationale for converting proprietary RAW files to DNG. Now, it appears I'm going to have to do just that.

It's not space that's the issue but the wonky relationship between Bridge CS4 and some of the RAW file formats, particularly Nikon's NEF. For some weeks now I've been unable to update the metadata in NEF files; I would continually get an error message when I attempted to update a group of files -- and, if this has ever happened to you you know what a serious hassle this is. On several occasions, when I tried to update a large group of files I would have to force quit Bridge since it wanted to show an error message for each file and each message had to be closed out manually. Not a pretty picture when you're trying to update 608 files. And this seemed to happen with each and every NEF file I was working on.

But this problem does NOT occur when working with DNG RAW files.

So, in addition to reducing overall file size (and consequently space concerns), I now have cleaner files. Or will have.

Since there are more than 8k RAW files in our central library I'll be at this for a while, to be sure. But I suspect it will be time well spent in not only making accessibility and searchability that more effective but also paving the way for archiving these image files for years to come.

Monday, August 30, 2010

JWU Central Image Library using Bridge CS4 is still working

It's going on nearly a year now since we created a central image library using only Adobe Bridge CS4 and so far, so good. There have been a few issues to be sure -- trying to modify metadata in the search results window is frequently of the more exasperating and seemingly irresolvable.

Right now we have more than 130,000 images -- still images only -- in the central library, which is accessible to our designers and writers.

There is, of course, the ongoing challenge of keeping up with re-caching (re-indexing) each machine. In other words, every time metadata is modified or added to any existing image in any folder, that folder has to be "re-cached" in order for the new metadata to be searchable.

This can be a challenge to be sure but is certainly not an overwhelming one. Each week I keep a running log of the folders where image metadata has been modified and the following Monday I send out an email noting those folders that need re-caching. It's then up to the designers and writers to make sure their machines are up-to-date.

If someone leaves on vacation or, as happened recently in our office someone is out on maternity leave, I make a point of re-caching that particular machine myself.

It's really that simple.

Next: Using the Collections Panel will streamline your workflow.

Monday, April 5, 2010

Using DNG File conversion to maintain control over Image Library Size

Out of more than 110,000 photos in our image library nearly 8,600 are Camera RAW files (NEF, CR2, DCR, RAF primarily). Although they make up less than 8% of the total number of files, they account for more than 12% of the total space taken up on the drive (68 gigabytes out of 526 gigs of space occupied by the library).

The challenge is how to preserve the integrity and optimize the size of the library, reduce the multiple file formats and standardize the image files, while acquiring the best quality images for use over the coming years.

Camera RAW files are the obvious answer but the file space these would require prohibit our organization from using RAW files.

Adobe's DNG file format system is widely touted as achieving smaller RAW files while protecting the integrity of the image data -- unlike with jpgs.

Moreover, DNG files also combine the metadata into the image itself unlike RAW files which use the sidecar method of storing xmp information, and the problem with sidecar files is you risk loss of that data if the sidecar file becomes separated from the image file.

Recently I undertook a series of in-house tests using 30 random camera RAW images from a variety of different RAW formats. Bridge's built-in DNG converter made the process simple and easy.

Results:

Orig RAW: 30 images (238.5 mb)
DNG convert: 30 images (181.8 mb) - with no loss of information

Moreover, the DNGs open up just like a camera RAW file in PS CS4.

Several options are now available:

All contract photographers would be required to send the "keeper" RAW files on disk, and we convert to JPG for uploading to the library (thus keeping the library at a reasonable size). If a larger, more robust copy of that image is needed, the metadata in the JPG will direct the designers to the specific disk where they can find the RAW file. (Professional digital photographers shoot predominantly in Camera RAW in any event.)

Another scenario would be to request that all contract photographers convert their final images to DNG instead of jpg before sending them to us. This would require no additional steps on their part, since they convert the images to something after color correcting, reviewing, etc. And we get the added benefit of the BEST quality image. Of course, the photographer could send us the RAW files and we can convert in-house. With Adobe's PS and Bridge the process is quick and easy, and we can even add our own metadata information during the conversion process.

Lastly, the photographer could send us BOTH the RAW and JPG files, since DSLRs today are capable of shooting images in both formats at the same time. No conversion is needed -- however this does not take into account those files that were color-corrected.

Wednesday, March 24, 2010

Should you migrate from Adobe Bridge to a Digital Asset Management (DAM) Program?

By now most of you know the difference between a browser asset manager ("organizer" would be more accurate) and a catalog asset manager. A browser program such as Bridge is inexpensive and generally easy to use. Search features in Bridge are robust and seem to become even more so with each successive iteration of the program.

But there are serious limitations to a browser program:

Each user must fully index and continue reindexing their Bridge cache locally, and the larger the library the larger each machine’s cache. For example, the cache for JWU’s image library runs to more than 24 gigs -- and that's on each and every user's machine.

After months of fruitless searching the message boards at Adobe and the Internet generally, it seems safe to say that no one really knows exactly where or when Bridge’s cache limit would be reached. Or what would occur when that limit is crossed.

According to Adobe’s user guide for Version Cue and Bridge (May 2009), the cache can contain up to 500,000 items (or properties), although it is unclear what exactly constitutes an “item” – is it all files in the cache, including sidecar files? Does this number include the previews and thumbs, thus cutting the number of images by half?

Finally, what happens when the limit is reached? According to the user guide the oldest items will be removed to make way for the new – is there a message informing the user of this happening? What will happen to the indexing and reindexing process?

Obviously, sharing the library to more than a dozen or so trained users is a very difficult proposition.

Additionally, with a browser you cannot work offline but must be directly connected to the original image source.

If you are working with a a few thousand images shared by only a handful of users such concerns would most likely not be troubling -- particularly if your user-base and image-base would not grow appreciably in the future.

If, on the other hand, your user-base is likely to increase, or if your library has tens of thousands of images and will probably increase by thousands more each year, then an alternative approach to sharing and using a central image library must be given serious consideration.

One obvious choice is, of course, a mature digital asset management (DAM) system.

There are two basic types of DAM systems:

1. Software as a service (SaaS), also known as “hosted provider,” “cloud computing” or “Application Service Providers” (ASPs). All hardware and software reside on the site of the SaaS provider and access is granted through a client branded login page at a customer specific URL. Examples would include Honeycomb, North Plains, OpenText, and DigitalAssetManagement.

2. Installed software. Provides application software and various hardware components, but internal IT typically installs and supports the system in conjunction with the software provider or a third-party implementation team. One example: Cumulus by Canto

These systems have their relative strengths and weaknesses to be sure. The crucial thing to know is that both types of DAM provide incredibly robust features that allow digital assets to be widely shared, distributed and modified by large numbers of users in just about any location.

But there is a price to be paid, literally.

Unless your organization has budgeted thousands of dollars for an enterprise strength asset management system, you'll probably want to take a long hard look at a catalog program such as Expression media from Microsoft.

Next, catalog programs.

Wednesday, March 17, 2010

Creating a Contact sheet or Web Gallery in Bridge

Photoshop evangelist Julieanne Kost of AdobeTV.com walks us through the new output features found in the output module in Bridge CS4. That's right, the old "Contact Sheet" command in Photoshop is gone, replaced by the incredibly user-friendly "Create a pdf" feature that allows quick and easy way to share a collection of images with a client.



Questions, comments or suggestions? Feel free to contact me: steve.soper@jwu.edu

Tuesday, March 16, 2010

Working with Metadata and Keywords in Bridge CS4




Questions, comments or suggestions? Feel free to contact me: steve.soper@jwu.edu

Monday, March 15, 2010

Chris Orwig introduces the new interface for Bridge CS4

Chris introduces the interface to Adobe Bridge and how to use the review mode to quickly scan through a large selection of images.


Questions, comments or suggestions? Feel free to contact me: steve.soper@jwu.edu

Friday, March 12, 2010

Introducing Adobe Bridge CS4




Questions, comments or suggestions? Feel free to contact me: steve.soper@jwu.edu

Thursday, March 11, 2010

Two concerns in using Bridge CS4 to maintain a central image library

Using Bridge CS 4 as a tool for creating, maintaining and accessing a large central image library has worked for our organization. After much testing in the summer of 2009 and continual testing and updating of our search and keywording strategies the library remains stronger than ever.

However, there are two ongoing concerns that one should be aware of: exactly what is the top end limit for the cache and what will happen when that limit is reached.

It is widely known that you can adjust the number of "items" in Bridge's cache preferences from 10k to 500k. But exactly what does Adobe mean by an "item"?

Elsewhere they use the word "record" but generally the word item seems to be most favored. For example, is a RAW image file and its sidecar metadata file considered two items? Is each thumbnail and each preview a separate item?

If the answer is yes to both questions, then it is safe to assume that one camera RAW image could in fact be read by Bridge as four distinct items. It wouldn't take Bridge long to reach the maximum threshold of 500k.

Second, exactly what happens when Bridge reaches its maximum number of items?

According to Adobe's user guide (May 2009) for Bridge and Version Cue, "If the cache is near the defined limit (500,000 records) or the volume that contains the cache is too full, older cached items are removed when you exit Adobe Bridge."

Here, again, we are in murky waters. Note the use of both the maximum number of records or if "the cache is too full." Exactly what does the latter mean and how does it differ from the former?

I have yet to discover the answers to either question -- perhaps none exist, at least not an answer that would cover any and all possibilities.

Any ideas, comments or questions, please feel free to contact me: steve.soper@jwu.edu

Tuesday, February 2, 2010

Update on using Adobe Bridge to access a Central Image Library

It's been several months now since the central image library went live. The images in the library are searchable by more than a dozen writers and designers in our organization and I can report that so far, so good.

We now have probably 120,000 images in the library and I add more every week. This requires a cache in Bridge of more than 24 gigabytes but we haven't noticed any general inability to find images in any given search. Seaching can be slow at times, depending on the complexity of the criteria selected and the number of images returned in the search results.

Remember, we are on new ground here: Adobe claims there is a limit of around "500,000" items but this is rather vague. For example, exactly what constitutes an "item"? Does that refer to each and every file type (such as sidecar xmp files for camera raw images)? How does one monitor those numbers? Also, how much space does the cache need to consume before the machine begins to slow down altogether? We just don't know.

The one singular challenge -- and it is important, believe me -- is to ensure that everyone's machine is up-t0-date in its indexing. What I mean is, that anytime a modification is made to a file in a given folder in the central library -- say metadata is added as information about a selection of images comes to light -- then that folder must be re-indexed by each machine in order for those images to be "searchable."

Also, I would suggest routine purging and rebuilding of the caches. Ideally, one might want to physically go in and remove the old cache and start fresh every few months or so. But this is not always feasible.