Winging It: How to gauge those travel evaluators

Memorial Day marks the time when many of us get serious about planning a summer vacation.

My planning usually starts by looking at websites that offer customer reviews of hotels, airlines, and other travel services. I also consider the plethora of press releases I receive throughout the year, announcing ratings for the travel companies, based on consumer surveys or experts' opinions.

After discounting some of the releases' hyperbole about the perceived importance of each survey, I'm still left with a key question: If one airline or hotel company receives high praise in one opinion poll or on one website, and a poor grade in a different review, what should I believe?

I recently found one hotel-review website, www.oyster.com, that is more useful and seems more trustworthy than some others (more about it in a moment). For the most part, however, only personal experience, or finding multiple sources of information about a travel service, gives me the confidence that I'm getting the best value for my money.

Before trusting what some surveys or rankings of airlines and hotels have found, remember this caveat. Most of the surveys are of a self-selected sample, made up of travelers who had the time or interest to participate. That makes the survey generally less reliable than a scientifically based, random sampling of all or some segment of the population.

That said, the sheer volume of voluntary reader responses that some websites get, such as the hotel-centric www.tripadvisor.com, with dozens to hundreds of reviews on thousands of hotels worldwide, can make them valuable resources.

Consumer Reports magazine, one of the more widely respected collectors of survey data, is another example, basing its product and service ratings on thousands of voluntary responses from its readers. Zagat's restaurant guides use the same methodology.

Another factor to consider is whether airlines, hotels, or service providers get good marks in customer surveys year after year.

The SmarterTravel.com site, for instance, chose JetBlue for its "Editors' Choice" award for most customer-friendly domestic airline, followed closely by Southwest. Those two consistently rank high in a variety of measures of customer service.

But Anne Banas, the site's executive editor, told me that her staff chose AirTran Airways as the "best value airline," despite Southwest's reputation for low prices, because AirTran displays more of its discounts to more places in an easy-to-use way on its website.

In the annual SkyTrax (www.worldairlineawards.com) surveys to identify the world's best airlines, the winners are usually based in Pacific Rim or Middle Eastern countries, where carriers' low labor costs allow them to have a high ratio of cabin attendants to passengers. Also, many voters in the poll are business travelers who can afford to sit in the front of the plane.

I've not flown on South Korea's Asiana Airlines, Singapore Airlines, or Qatar Airways, the top three vote-getters in this year's SkyTrax poll, but I am ready to give any of them a try since they have scored high for several years in a row.

Choosing a hotel for a long stay is often the most challenging part of trip planning simply because there are so many of them and their quality varies so widely.

Conde Nast Traveler magazine publishes a "Gold List" of the world's best hotels, resorts, and cruise ships, based on an annual survey of its upscale readers. But most hotels on the list cost $400 or more a night, so it's of limited value to many people.

The www.tripadvisor.com site, which I generally like, has been criticized as unreliable. Hotels have claimed it can unfairly damage reputations with unfounded allegations, and hotels have been accused of planting glowing reviews written by staff members.

The criticism has subsided recently after the site allowed hotels to respond to complaints posted by guests, and steps were taken to catch phony reviews from nonpaying customers.

The www.oyster.com site appealed to me because it has only detailed hotel reviews by professional writers who paid their own way and were paid for their work. Senior editor Will Begney told me that the site makes its money as a neutral hotel-reservations booking engine.

"We go above and beyond by being honest about the hotels," Begney said.

Readers can decide, he said, "Is it really worth upgrading to the ocean-view suite over the standard room?"

Oyster does have a serious drawback: It covers only 15 destinations, including seven major U.S. cities and resort areas in the Caribbean and Hawaii. It has reviewed about 900 hotels, compared with tens of thousands on TripAdvisor and online travel agent sites such as Expedia, Travelocity, and Orbitz.

If this vast river of travel information available online still leaves you confused, there are always some old-fashioned ways to find help.

You can consult with a human travel agent, who will probably charge a service fee but also should have expertise about particular destinations and travel suppliers.

And there's always the bookstore, where you can browse travel guides to get a true sense of the places you may want to visit.