Are Corporate Web Sites Optimized for SEO?

Nearly every business has a Web site these days. From the local paint store to the major corporation, the Internet is home to millions of Web sites with billions of pages. But are most of the corporate Web sites being found in the first three pages of major search engines? Not yet, according to recent research.

• In a study of national retailers, Oneupweb found only 12 out of 100 sites were well optimized. This made a marked difference in their respective search engine rankings. Well-optimized sites got top listings that yielded extraordinary increases in traffic and conversions. However, the non-optimized sites were not found in the first three search engine results pages (SERPs).
• Another Oneupweb study of Fortune 100 corporations yielded similar results. Well-optimized sites got the top rankings that dramatically improved their bottom line. However, less than 10 percent of America�s top corporate Web sites were optimized to reap these rewards.

While the above research was based on 2003 and 2004 data, the same trend was reported in a 2005 study conducted by Fathom SEO on manufacturing sites.

• Content on the majority of the Web pages was not properly optimized with strategic keywords.
• Only 25 percent of the manufacturer�s homepages had effective Title Tags that used strategic keywords. The internal pages fared worse — 20 percent had Title Tags. (A Title Tag is an important element in search engine ranking algorithms because the text is generally used as the link users view and click in the SERPs.)

Only 10 to12 percent of corporate Web sites are optimized and well positioned in the organic search results. What happens to the rest of the sites buried beyond page 3 in the SERPs? They bear a heavy lost-opportunity cost that dwindles the bottom line.

Search Marketing Comes of Age

Search Engine Marketing (SEM) revenues are growing at a faster rate than any other marketing strategy. In May, Forrester reported that SEM would grow by 33 percent in 2005, reaching $11.6 billion by 2010.

SEMPO research shows that advertisers intend to spend about 39 percent more on SEM programs in 2005 over 2004.

While that sounds promising, the bulk of the SEM revenues went to paid placement (sponsored listings), not organic SEO. In 2004, 82 percent of SEM expenditures were allocated for paid placement ads, versus only 12 percent for organic SEO. Similar patterns are predicted for 2005.

Half of the marketers surveyed said senior management was very interested in SEM programs and considered them a high priority. Another 36 percent said their senior executives were moderately aware and interested. What�s necessary is to educate senior executives on the effectiveness and long-term durability of organic SEO, in addition to its branding capability and excellent return on investment (ROI).

What Defines Organic SEO?

Organic SEO consists of technical and editorial Web site optimization services to improve your site�s communication with search engines. Major tasks include a site review of back-end items, search-appropriate HTML coding, linking strategies and SEO copywriting of major pages. Also recommended are competitor analysis and Web analytics. The goal is to provide the necessary elements to attract search engine spiders and acquire high-ranking listings in the major search engines like Google, Yahoo! and MSN. Once a site is optimized correctly, its organic results are long lasting with periodic maintenance.

Not only are organic listings longer lasting than paid listings, they�re more valuable. Research shows that 70 percent of search engine users will click on organic listings, while the remaining 30 percent click on paid listings. Another study found that 6 out of 7 sales resulting from search engines originated in the organic search listings.

An excellent virtue of organic listings is their marketing accountability. It�s easy to track unique visitors, increased conversions and ROI. With a good Web analytics program such as WebTrends, ClickTracks or WebSideStory, you can take baseline measures before the optimization campaign and then again a few months later to see the initial gains. Continual monitoring and adjustment at the beginning of a campaign is also recommended.

Here�s what you need to know before conducting a search engine optimization campaign, starting with your budget allocation.

Funding Your SEO Program

SEMPO research shows that only 43 percent of advertisers created new budgets for organic SEO, and these were mostly large companies. Most advertisers shifted money from existing marketing programs to fund SEO or used a combination of new funding and shifted funds. Below are a few ideas to help your funding efforts.

Influence Your Decision-Makers: Convince your high-level decision makers that organic SEO is an essential marketing strategy for improving the bottom line. Get a senior-level manager on your side by providing them with the data and research to convince their peers.

There is ample research showing that SEO generates more leads and conversions when compared to other marketing strategies. Organic SEO is cost-effective compared to TV advertising, radio ads and print (give me an email or phone call if you want me to send you the data). It�s just a matter of finding the right person to influence your decision makers.

Identify Your Competitors: Find someone qualified to provide you with a competitor analysis, identifying the competitors that outperform your company�s Web site rankings. This is usually very motivating in itself and often leads to opportunities for improving your company�s Web site and conversions. Calculate Your Lost Opportunity Cost: Another persuasive tactic is to estimate the lost-opportunity cost. You can use WordTracker or the Yahoo Search Marketing Keyword Selector Tool to learn the number of searches for your company�s strategic key terms, and then estimate the number of likely unique visitors and conversions that are being missed because your Web site is not listed prominently in the major SERPs.

Setting Your Site Goals

It�s very important to establish and prioritize your site goals, be they branding, increasing sales or generating leads. SEMPO research shows that most companies want to increase brand awareness. Other goals specified were: selling products/services online, generating leads, increasing traffic, generating leads for distributors, and providing information/education. You can establish primary and secondary goals of this nature.

It is also important to know and understand your customers so you can optimize for the key terms used to find your site. There are a number of ways you can get feedback from customers, like requests for feedback when they take action on your site. Another great resource is to have a site search function. You will find out what your customers are looking for, and this will supplement the info from your server logs and other keyword analysis tools when identifying your relevant key terms.

Selecting an SEO Agency

A reputable SEO agency can help optimize your site for better rankings. Most agencies will offer a caf�-menu of SEO service options. Reputable firms will also provide reports on traffic and conversions, per your specifications. When selecting an agency, you should clearly define the services that will comprise your SEO campaign. Such services can include:

• Competitor analysis
• Technical site review,
• Complete keyword phrase research
• Body text edit/copywriting
• Title tag optimization
• Description tag optimization
• Manual submission to search engines and directories
• Paid inclusion
• Linking strategies
• Creation of new pages
• Paid search options
• Reporting and tracking

What Spells Success?

It used to be that high rankings and increased traffic were the major goals of search engine optimization. Currently, ROI and increased conversions are of more interest to marketers. Whatever your measure of success, it�s easy to document with organic SEO.

Search Plays Key Role in B2B Sales — Part 2 of 2

In part 1 of this article, we talked about the uniqueness of B2B marketing and the dominant role of search engines in the B2B buying process. So it�s safe to assume that search engine visibility is an excellent way to boost B2B profitability.

Now it�s time to explore the return on investment (ROI), as well as the effectiveness and flexibility of search engine marketing, focusing on organic site optimization.

How Good Is Search Marketing ROI?

We all know that ROI can be difficult to measure. In a survey conducted by the Association of National Advertisers (ANA) and Forrester Research 78 percent of respondents said, “measuring the sales impact of marketing was difficult.” In fact, most respondents could not agree on the definition of ROI. Here�s the ROI definition preferred by 66 percent of respondents: �Incremental sales revenue generated by marketing activities.� The ROI for B2B websites using an organic search engine optimization program can go through the roof, shaming just about every other component of your marketing plan. Why? It�s simple.

We all know that B2B Internet users prefer organic search results for reliable information. It�s been reported by dependable sources such as SearchEngineWatch, MarketingSherpa, and eMarketer.

We all know that properly optimized sites get good rankings on Google and other search engines. There is ample industry research and case studies showing those high rankings on page 1, 2, or 3 result in extraordinary increases in unique visitors and sales conversions for B2B clients.

We also know the fantastic value and loyalty created when B2B�s provide information during that long sales cycle. Engineers just love those white papers and webcasts offered as a grab �n go freebie on B2B websites. The technicians and specialists doing research really do appreciate your downloads, and 70-80 percent of them return and convert from visitor to user.

However, it�s important to have your marketing performance measurement tools in place. MarketingSherpa reports that 90 percent of CMOs consider these measurement tools a �high or moderate priority.� With the many web analytics tools available for search engine marketing it is easy to get a benchmark before your campaign starts, then track ROI as your rankings improve.

How Effective Is Search Engine Marketing?

As documented in many case studies, search engine marketing is quite effective at increasing site traffic and conversions. Below are two examples of B2B organic search campaigns from Sempo.org.

OneUpWeb reported that an international manufacturer and distributor of secure fasteners initially achieved 48 top-10 positions and 52 top-30 positions, starting off with 0 before its search engine optimization campaign. One year later, it went to 130+ top-10 positions and 250+ top-30 positions. All new sales leads come from search marketing efforts, many from outside the U.S., boosting international sales by 50 percent.

Pennant reported dramatic results with search engine marketing for a connector manufacturer. Starting with a baseline of 15 top-30 rankings, it went up to 79 in 6 months, then 102 in 4 more months. The site is in the top 9 on Google for specific phrases and increased highly targeted visitors three fold over baseline in one year.

How Flexible Is Search Engine Optimization?

Want a marketing strategy where you can pick and choose? SEO pricing is caf�-menu style, so a business can select only the services it needs or can afford. That�s why search engine optimization was referred to as “� the ultimate cost-effective, online guerrilla-marketing tactic that generally has the highest ROI of any type of Internet marketing” by Marketing Sherpa.

When you select a search engine optimization provider, you should clearly define the services to be included in your SEM campaign. Here’s a sampling of what you might consider:

• Consulting, managing, coordinating the SEO plan, outsourced or with in-house IT
• Web analytics: verification, installation of traffic reporting, comparison, ROI calculation
• Site Analysis of architecture communicating with search engines like Google
• Keyword Research, determine relevancy and traffic predictions
• Competitor Analysis, comparison with your top contenders
• Elevate and Control Brand: organic optimization, SEO copywriting for keyword list
• Enhance Inbound Links: Search engine optimized Press Releases and RSS feed
• Research industry-specific search engines to match your company website objectives
• Review current Yahoo! and DMOZ listings, update categories, titles, descriptions

There�s one more thing B2B marketers might consider, and that�s whether to optimize in-house or outsource to a search engine optimization firm. Large corporations may prefer in-house optimization, but outsourcing can be a viable option.

In a Nutshell

It’s wise for B2B marketers to include search engine marketing in their marketing budget. The return on investment goes a long way toward increasing traffic, conversions and branding. SEO is too valuable a marketing tool to ignore.

Search Plays Key Role in B2B Sales � Part 1 of 2

Conventional wisdom tells us the fastest and most efficient way to research and compare products and pricing is on the web. Recently, this belief was confirmed by two research studies examining the role of search in B2B marketing.

First, Enquiro and Marketing Sherpa demonstrated the dominance of search in B2B transactions. Now, a Google-Millward Brown study corroborates the importance of search in the B2B sales cycle.

B2B Transactions Are Unique

As you know, B2B transactions differ from consumer transactions because the decision requires coordination among several personnel before the final sale is made. The time between researching the product and placing the order results in a long sales cycle.

Enquiro’s “The Role of Search in Business to Business Buying Decisions” is a well-designed study of approximately 1,500 participants who responded to a 40-question survey about B2B buying habits. You can download the entire report for free. Below are a few highlights:

• When participants were asked to indicate how they would go about making a B2B purchase, 93.2% said they would research the purchase online.
• When asked if they would use a search engine at some point in this task, 95.5% of participants said they would.
• When asked where they would start their search for information, 63.9% of participants chose a search engine over consumer review sites, e-commerce sites, manufacturer’s sites, and industry portals.
• When taking budget into consideration, manufacturer’s sites and industry portals were the chosen starting place as budgets increased. However, 86.9% of participants said they would visit a search engine after visiting those sites.

Search Engines Dominate

The study is too rich with details to cover entirely in this article. But here are some important conclusions:

• Search engines play a dominant role in B2B purchases.
• Search engines are used in the early or mid research phase in the buying cycle.
• Google is favored over other search engines.

� Search engine research takes place at least one to two months before the buying decision.

• Good balance between organic and paid search is necessary. Organic links get over 70% of the clicks.
• Position is a factor, with over 60% clicking on the top three listings.
• Most users decide which listing to click on within seconds of scanning the page.

B2B Buyers Prefer Search to Traditional Media

The Google-Millward Brown study examined technology purchases to identify sources of information used during a three-stage buying cycle defined as �research/engagement,� �consideration/comparison� and � final purchase.�

The study found that B2B buyers used search engines more often than traditional media throughout the entire buying cycle. Google concluded that search advertising is more effective than trade magazine ads and other traditional B2B media.

• Research phase: Search used 30% more often than trade publications.
• Consideration phase: Search used 21% more often than B2B publications.
• Final purchase phase: Search used 62% more often than traditional media.

The study also found that benefits and strong call-to-action are crucial components of a sponsored link (duh!). This has implications for the title and description of your web pages as these elements are used to create organic links on the search result pages.

Are B2B Sites Optimized for Search Engines?

They all have websites, but according to research, most aren�t optimized to maximize search engine traffic. Research indicates that 80 to 90 percent of corporate websites are not well optimized. Most recently, Fathom reported manufacturing sites ignore this marketing opportunity. Yet, this same research shows that properly optimized websites enjoy fantastic gains in unique visitors and conversions.

With all the qualified traffic originating from search engines, it is more important than ever for B2B marketers, wholesalers and B2B exchanges to ensure that their web sites are correctly optimized for good positioning in search results. There is also great value in search engine optimization as a marketing tool.

It�s a User-Friendly Tool

Search engine traffic is highly targeted. That’s because potential buyers who find your B2B offerings through search engines are looking for your products and services on their own; as a result, they are predisposed to hear your marketing message. You can’t find a more qualified prospect than that.

Here’s what distinguishes search engine marketing from other types of advertising:

1. It is non-intrusive. Search marketing is a non-intrusive marketing tool. Most advertising, both online and offline, interrupts consumer behavior. If a user goes to a web site for info, up pops an intrusive ad. Reading a newspaper? Ads dominate and force articles to be continued on another page. With search engine marketing, the user is actively seeking your products, services, and information. They are delighted to be driven to your site.
2. It is voluntary. Search marketing is the result of user-originated behavior. Your visitors from search engines and directories have voluntarily clicked on your listing rather than any competitor’s; thus, they are motivated to explore your offerings.

What Else Do B2B Marketers Need to Know?

How good is the ROI? How effective is search engine marketing for B2B firms? What are the key-points to consider for your B2B search engine-marketing plan? Please join me next week for the answers in Part 2.

The Importance of an SEO Site Review

The Internet is home to millions of websites. Google claims to index over 8 billion web pages, and MSN announced 5 billion not long ago. Yahoo! is probably close; but upon query, a spokesperson said, ��we don’t disclose the size of the index, our top priority is to provide the most comprehensive and relevant search experience to consumers.�

How many users visit these websites? In the U.S. alone, search engine users totaled over 130 million in 2004 and will go beyond 150 million in 2006. If you�re in business to sell, there�s a captive audience out there.

Many companies have a website, but the majority of these websites are not optimized to maximize search engine traffic. Research indicates that 80 to 90 percent of corporate websites are not well optimized. It also shows that properly optimized websites enjoy fantastic gains in unique visitors and conversions. These huge gains have been reported a number of times in research and case studies. So why aren�t more companies reaping the rewards of search engine traffic?

What�s Wrong With the Average Website?

Many companies are neglecting the important back-end issues that can make their websites search-engine friendly. Sure, users visit these website and click to download white papers. The graphics are sweet, and some of these sites rock. They may not have the best navigation and usability, but landing pages are in place � it all looks first-class. When we surf the web, we see a lot of this same functionality. But the problem with the average website is that it�s not optimized to gain top rankings on the major search engines.

While all appears to be smooth on the front end, potential problems lurk on the back end. To quote Dan Thies of SEO Research Labs, “We routinely conduct technical reviews for our SEO partners, and find duplicate content or other server-related issues on over half of the sites we see.”

Watch Your Back End

There are a plethora of back-end technical connections going on behind the scenes, and if you�re lulled into complacency, your site could be missing the road to top rankings that increase your conversions.

When it comes to communicating with search engine spiders, many of America�s best-known company websites are not properly optimized to gain top rankings. That�s because the back-end is shorting-out and not connecting with the stealthy search engine robots that index and rank websites for major search engines like Google, Yahoo! and MSN.

Following is a list of back-end technical items related to search engine robots that must be addressed if you want your company website to perform well and outperform your competitors in the major search engines. As part of your Organic Search Engine Optimization Program, the items below should be thoroughly reviewed, and appropriate recommendations made, in a professional SEO Site Review.

1. Site Architecture and Applications — This deals with your web server (e.g., Apache, Microsoft IIS, Sun) and the program languages used on your site (e.g., HTML, asp, php, jsp). A look into your web hosting (e.g., dedicated or shared) and your database technology (e.g., MySQL, Oracle, PostgreSQL). It also covers investigation of your web applications such as content management systems, site search, directories, and shopping carts. Consideration for how you set-up your cookies and sessions (human vs. spider). A check of your HTTP response headers looking at proper use of 404 error pages. Complete review of basics such as your contact information, FAQ, privacy policy, various resources, site map and number of links on a site map page.
2. Site Navigation – A look at your image maps and image links, javascript and DHTML and how they are organized on the page. Consideration for your text links and form based navigation, redirection 301, 302 pages and META refresh. A look at the effectiveness and placement of your global navigation and functional sections such as news, forums, shopping, etc. A review of site search functionality, usability and logs.
3. Document Structure – A review of your title tags, META description and META keywords. A look at your page layout CSS and tables. Proper headings (H1, H2 and page structure). The effect of pop-ups and browser compatibility such as AOL, IE, Mozilla, Netscape, etc. The effect of any plug-Ins, Flash or Java, scripts and stylesheets.
4. Keyword Strategy & Copywriting – An analysis of your primary themes and search terms, your supporting keywords and their optimization and effectiveness in your current content. How you are using landing pages and their URLS. A look at your internal links (cross-linking) also related to the use of keywords and their placement. A look at keywords and placement in outbound links.
5. Search Engine Positioning – Index saturation: the number of pages found and/or indexed in Google and the crawlability of the site. Identify potential duplicate content and review link popularity, link quality and relevance. Check PageRank on Google toolbar and directory listings such as DMOZ and Yahoo.

The Road to Top Rankings

All search engine spiders must first be able to digest your content before they can post your high-ranking results. That�s why a technical site review is the first step in Search Engine Optimization. Once the above back-end items have been analyzed by a qualified SEO technician, your IT people can begin the process of making the necessary adjustments to your website. When adjustments have been made, you are on the road to capturing all of that delicious organic traffic. Happy traveling! Remember, it�s a journey not a destination.

All Aboard! Inbound Links for SEO

Did you know that search-engine-optimized Press Releases can help you achieve excellent rankings within major search engines? SEO press releases provide opportunities for inbound link development when Yahoo News, Google News and major news search engines list your data online.

The link structure of a properly distributed �search optimized� press release also benefits your website when you issue a release because you are providing relevant information to those searching for news related to your industry. It could be a prospect looking for industry info when researching a purchase or a journalist researching a story. At any rate, your company will be represented in the search results, which increases the relevant traffic to your site.

SEO Press Release deliverables will generally include:

• 450-500 word press release; optimized for a core search term and 2 to 3 niche terms
• Release incorporates anchor text; based on search term associated with target web page
• You often have your choice of distribution
• Access to view statistics about how your news release performed over a 30-day period
• RSS feed and paid inclusion distribution

Keyword Analysis

Prior to preparing your search-optimized press release, a qualified SEO technician conducts keyword research and planning. Using the business information you provide, the vendor can then make comparisons with your competitors and similar websites. A weighted value is calculated for each keyword based on the number of searches over the past two months compared to its percentage of relevance.

Because the first step in optimizing a web page for search engine ranking is placing keywords in the Title, it makes sense to determine how many pages have been somewhat optimized. A good vendor will perform an �InTitle� and �Title+Anchor� search at Google to give you an indication of the popularity of your various terms, comparing these results with WordTracker�s KEI (keyword effectiveness index).

This research allows one to determine relevancy, evaluate competition, and add modifiers for providing you the best advice on selecting your best keywords and phrases. The vendor may also marry the Keyword List to a Site Review and consider your current content to make recommendations for overall improvement and performance.

Benefits of an SEO Press Release

� Top rankings in Google News, Yahoo News and MSN news for targeted search terms � New rankings in Google and Yahoo web search results, often first page results � Coverage on hundreds of relevant web sites subscribed to PR Web�s RSS feed � Links from web sites that pick up your news release � Help in building link popularity for your web pages � Archived press releases to deliver long-term link value

Often, the vendor can provide you with a tracking report to summarize the rankings achieved in news and web searches. The vendor can also provide stats on top search terms for which a news release is accessed; page reads and downloads of a release; number of sites that carried the release as news; and number of links created back to target web page!

There are several such vendors, and currently I prefer using SEO-PR. As noted on their website, �Greg Jarboe and Jamie O�Donnell, the co-founders of SEO-PR, have 40 years of combined experience managing PR for tech companies and PR for web companies.� So you�re in good hands with these guys.

FAST Breaks the Flash Barrier

Fast Search & Transfer (FAST) is the first search technology company to enable indexing of Flash content in AllTheWeb and partner sites. It probably won’t be the last.

In July, I wrote about the push to index file formats beyond HTML, showing that search engines were indexing formats such as Image, Video, MP3, and PDF files, and demonstrating how these files can be optimized for search engines. I predicted FAST would eventually index Flash and other multimedia files.

Well, it didn’t take long. FAST just announced AllTheWeb and its partner sites will now index Macromedia Flash content and applications, a breakthrough that allows many popular sites to get expanded visibility in search engines.

As the Web matures, more and more designers and Web site developers are using Flash to create Web sites that go beyond the limits of HTML. That’s because users are demanding more sophistication with animations, multimedia features, and advanced menu options. In fact, Macromedia Director Param Singh states that “Macromedia Flash content is included in nine of the top ten most visited websites.�

Up until now, it’s been difficult to get Flash content indexed. We’ve advised our clients to use Flash along with HTML text links to make a page crawlable by search engine spiders. This also makes Flash pages readable by users who don’t have a Flash plug-in, as many users are still resistant to downloading plug-ins.

While a few search engines (like Google) include some Flash sites in their databases, these pages are not regularly indexed by search engine crawlers. So the bulk of Flash pages do not appear in search engine databases unless they also include optimized HTML.

Expanded visibility for Flash sites is good news for designers and users alike because it means that AllTheWeb, Lycos, InfoSpace, and other partner sites using FAST technology can readily display links to a cache of Flash content that wasn’t available to users before.

Advanced Search Options

AllTheWeb alone serves over 100 million users per month. Its users can now further refine their searches in Macromedia Flash content and applications through the use of the site�s Advanced Features section.

There are a couple of new options in AllTheWeb’s Advanced Features section:

• A new Embedded Content option where you can specify whether to include or exclude embedded content in such files as: Images, Audio, Video, RealVideo & RealAudio, Macromedia Flash, Java applets, JavaScript, and VBScript.
• A new Result Restrictions option where you can specify the document depth at the directory level when querying your search terms. That is, you can ask for search terms that appear at the home page level and above or below, all the way to the 10th level directory of the site architecture. Alternatively, you can search “all document depths.”

�This new, advanced search feature from FAST presents a winning solution for the millions of people looking to perform highly-specialized searches for relevant information contained with Macromedia Flash content on the millions of sites which include our technology, said Macromedia’s Singh. It seems to me that these new advanced search options can certainly improve the user experience — provided that ordinary users become aware of them and use them.

Flash Q&A

When we first heard about this new functionality in our office, Operations Manager Nicole Falsey wasn’t sure she got it right. So she queried Peter Gorman, Director, Corporate Communications for Fast Search & Transfer, who answered her question thusly.

Question: Is FAST able to go into a site’s embedded .swf files to crawl and index the text within that file just like search engines do with HTML documents?

Answer: Yes, FAST crawls links as they appear within the document and treats Flash files like HTML when converted. FAST uses the Flash Search Engine SDK, which basically converts the Flash app into a HTML file. Click here for detailed information on the Flash Search Engine SDK.

Nicole immediately downloaded the Flash Search Engine SDK from the www.macromedia.com site and was able to convert .swf files into html. Not only does this tool allow FAST to index Flash .swf content, it may allow us to analyze our clients� Flash files and assist them in optimizing their sites. This will become important as other search engines follow FAST�s lead in indexing Flash content.

Converting .swf Files to HTML

Once she downloaded the Macromedia app, Nicole ran it on a couple of .swf Flash files downloaded from Macromedia’s web site. Below is a screen shot of one of those files so you can see the links and text that are viewable through a browser.

Now, when you run the swf2html application from a DOS prompt and type in the command to convert the .swf file to HTML, you get the following, which shows that there are other .swf files embedded within the first .swf file, plus text links and regular text in the first .swf file. Since many .swf files are embedded in each other, we’re assuming FAST would just keep converting all of them until it finished and had a completely indexed site.

We wondered how Fast pulls the TITLE and description for the search engine listings for Flash pages. We assumed these elements would come from the original HTML document’s <TITLE> and meta description tags, simply converting any Flash on the page to HTML, with this data added to the index in addition to the HTML page content.

But what if the Flash site is not embedded in an HTML document and is just a .swf file? Where will Fast get its listing title and description?

We got answers from FAST engineer Rolf Michelsen, who confirmed our assumptions were correct. “When the Flash file is embedded as part of a HTML document, we use the document title and various heuristics to extract title and teaser for our search results. The heuristics for extracting a teaser may use the meta description tag if present,” said Michelsen.

“When indexing a stand-alone Flash file, we extract title and teaser directly from the Flash file — basically trying to compose a teaser from the first few sentences of text extracted from the Flash file,” concluded Michelsen.

So there you have it from the horse’s mouth. This makes the critical point that designers should always embed their .swf Flash files in an HTML document and add a <TITLE>, meta description, and meta keywords tags to ensure their Flash based pages will be indexed in the search engines with a title and description of which they approve. Otherwise, the listing may show �No Title,� and the description will be the first text indexed in the file, which may not be advantageous.

Do You Need a Search Toolbar?

A number of search engines provide search toolbars that let you search from any Web site without having to make a trip to the search engine. A recent entry is Teoma, a free navigational tool that lets you search Teoma’s database from any Web page, providing convenience and useful information at your fingertips.

The most popular of these tools is the Google Toolbar, but several other engines also provide free navigational aids for loyal users, including Alexa, Excite, Lycos, Teoma, and Yahoo!. It’s possible to download several toolbars if you wish, one will simply appear below the other. Or try the new UltraBar, which allows you to search your own menu of search engines from a customizable toolbar.

Below is a summary of the various toolbars available and their features. Excite is an old one that goes on the desktop, probably designed with “push technology” in mind. Back in 1997, PointCast introduced “push” marketing methods for sending information directly to computer desktops, to one prospect at a time — counter to the traditional “pull” techniques of broadcasting new information to thousands, waiting for them to take the desired action. The desktop was rumored to replace the browser, not.

Toolbar Utilities for Browser and Desktop

The Alexa Toolbar enables you to find useful information about the site you’re currently viewing very quickly. From “Site Info,” you get contact information, site stats and user reviews at your fingertips. “Related Links” is a good way to discover similar sites. This list is updated constantly with information about where other users visit. There’s a link to the Google Toolbar and Dogpile. You can use the “Search” button to search from Alexa, Yahoo! and Google. If you can’t access your favorite site or want to see what it looked like a while back, use the “Wayback” button to see archived copies in the Way Back Machine.

The Excite Assistant, still available at download.com, gives you a toolbar on your desktop from which to search Excite and access a variety of online information such as stock and news updates. Listen to Internet radio and get personalized news without having to open your browser. Available for Internet Explorer or Netscape Navigator.

The Google Toolbar for Internet Explorer downloads very quickly and is probably the most widely used (several million downloads). This toolbar gives you a Google search box in your browser so you can access Google’s popular database from any Web page. Search the Web, the site, or the Google Directory. You can also access “PageRank” for the site’s link popularity. “Page Info” gives you a cached snapshot of the page, similar pages, backward links, and will translate foreign pages into English. “Highlight” will highlight your search terms in different colors as they appear on the page. “Word Find” lets you find your search terms as they appear on the page. Google collects info when PageRank is enabled, but not if you’ve selected the option, “Install Without Advanced Features.”

Lycos SeeMore for Internet Explorer 4 or 5 is a little different. It lets you search Web pages quickly by simply right-clicking on a word, phrase or graphic. You’ll get a menu from which to select “See More With Lycos,” then the utility will come up with more information links on your topic, allowing you to click for more information. For instance, if you wanted more information on “airport security” while reading a page describing a recent airport security breach, it will refer you to several text links from which to select relevant information. It can be handy because you don’t lose your place by going to a search box.

The Teoma Search Bar for Internet Explorer gives you instant access to Teoma’s search technology from anywhere on the Web. “Results” provide links to relevant Web pages, “Refine” gives suggestions for narrowing your search, and “Resources” provides links to specialized directories. It downloads and installs quickly, appearing every time you open a new Internet Explorer window. Features include a dictionary look-up, and a highlight button that will highlight your search terms in six different colors as they appear on the page.

Yahoo Companion puts a personalizable toolbar in your browser, allowing you to customize with buttons for Yahoo! services, saves your bookmarks for access from any computer, provides alerts on Yahoo! Mail, and allows you to search Yahoo! from any site. Available for Internet Explorer or Netscape Navigator.

UltraBar IE Toolbar is probably the handiest because it allows you select your most frequently used search engines from a menu within one toolbar, and it incorporates the latest search toolbar features. You can customize the order of search engines, highlight search terms within the page, and go directly to your search terms or phrases. It’s not limited to major search engines, you can select any site that has a search function. There are three versions, and two are free (Ultrabar IE Toolbar and Ultrabar Co-Brand). Ultrabar Pro is for ISPs and larger Web sites, offering added features and starting at $500 per year for unlimited downloads/installs.

Why Use a Search Toolbar?

Search toolbars are convenient because you can search from any site, saving the time it takes to go to a search engine and place a query. The newer toolbars offer additional information by highlighting your search terms throughout the page. With Lycos SeeMore, you can also link to relevant information from the text you are currently reading. For the times when you need to search multiple search engines or sites, UltraBar gives you complete flexibility.

While there are many positive benefits of the above tools, a major shortcoming is that all but the Excite Assistant and Yahoo Companion are available only for Internet Explorer, and none are available for Macs.

For Macs Only

There is a built-in toolbar in the IE Mac browser that can search MSN, Overture, AltaVista, Lycos, Excite, Yahoo, Euroseek, Bigfoot, Infospace, Worldpages, MSN Yellow Pages, Expedia, Mapquest, Encarta, Merriam-Webster, Dictionary.com, Thesaurus.com, and Corbis.

Additionally, Apple provides Sherlock 3, which can search the Internet in different “channels” of information for you. Most major search engines and directories offer plug-ins for Sherlock.

Optimizing Your PDF Documents

As search engines begin to crawl many different file formats besides HTML, you might consider optimizing your PDF documents for better search engine visibility. Many search engines now include image, video and PDF files, as well as spreadsheets, PowerPoint presentations, and more.

• Google searches 12 main file types in addition to HTML, the most common being PDF, PostScript, and Microsoft Office formats.
• FAST AllTheWeb indexes images, videos, PDF, MP3, and FTP files and eventually plans to index Flash and other multimedia files.
• AltaVista recently expanded its multimedia index to include news photos, maps, charts, and diagrams, as well as image, audio and video files.

Search engine users and researchers are finding a lot more PDF files than ever before, which can be useful for those looking for government and technical documents. However, there can be problems when accessing the various file formats as many of these files were not prepared with search engine robots in mind.

Limitations of Alternate File Formats

While it’s nice to have many different file formats available, they may not be as convenient to access as HTML. Some file formats aren’t always safe to access. They can take time to download and require special software. Below are the limitations.

• Slow Download Time: Users gathering information are in a hurry. Yet PDF files can be lengthy and contain numerous graphics. They can be hard to read on the screen. Users with dial-up modems may balk at the time it takes to download. This is also a problem with audio, video, and Flash files.
• Software Requirements: Users may not take the time to download the special plug-ins required to view your documents. Some users with older operating systems and software may have problems with current word processing and spreadsheet documents.
• Virus Risk. Along with the convenience of alternate file formats comes the vulnerability to pernicious new viruses that might be lurking in these files. You must have adequate virus protection and also exercise caution when downloading or opening files from unknown sources.
• Security Hazards. Confidential company data in spreadsheets and other files could be vulnerable to outsider curiosity, creating a security breach. Protect your confidential data with password protection on a secure server. You can also create a robots.txt file to disallow search engines from indexing specific files and directories containing confidential data.

Google has anticipated some of these problems and provides an option to view the documents as HTML. Users selecting this option can view the document in HTML (without formatting) on Google’s server. This gives you a rough idea of the document content so you can decide whether or not it’s worth downloading.

Optimizing Alternative-File-Format Documents

As the major search engines show more and more documents in various file formats in their search results, it’s a good idea to start optimizing the alternative-file-format documents that you want indexed. Don’t include those that could cause user frustration (use the robots.txt file).

Title and Description Tags: A good way to start is to ensure that you have a meta title and description tag for every file. The document title is one of the first things search engine spiders encounter when indexing your pages. If they don’t find one, they’ll create one that may not be as advantageous as your own.

Sometimes you can embed meta information that is read by search engines when creating your files in the various file formats. For instance, MS Word lets you select a document title when you create the file. If you don’t select a title, the first words to appear can be saved as the title, but this may not be your best choice. Keep in mind that title selection is possible in MS Word, PowerPoint, Adobe Acrobat 5.0, and other software programs.

It’s wise to provide a good description for PDF files because many users hesitate to download them unless they really want the data. This can be done by creating an HTML version, which can then be converted to PDF. There are free tools available that will perform the conversion for you. HTML Doc 1.8.20 from Easy Software Products is an example of such tools, but there are others. This way, you can create a good title and description that will be picked up by search engines while taking advantage of PDF format for document delivery.

Optimizing Image Files: Another good strategy is to index your image files, which is simply another good opportunity to promote your site without much extra effort.

• Image File Names: Be descriptive when assigning file names to your image files, using keywords if possible. This helps with general maintenance and can also boost your keyword density.
• ALT Tags: When you write text descriptions for your graphic images, make it meaningful and use complete sentences with strategic keywords rather than list a string of keywords.
• Page Content: The content surrounding the images on the page should also be keyword-rich and relevant to the image. This helps search engines put the image into context as the robot spider reads surrounding text when indexing images. So integrate your ALT tag descriptions with the surrounding page text.

In order to get more of your content indexed, you might give serious consideration to identifying which alternative-file-format documents to optimize. Go over the limitations and decide whether or not the download is worth the effort. To avoid user frustration, ensure the document contains valuable, current information and optimize with relevant descriptions.

The FAST and Fresh Machine

Search technology is rapidly evolving these days, and Fast Search & Transfer (FAST) ranks among the best with its AllTheWeb public search engine. Like many search engine companies these days, FAST’s business strategy is two-pronged. Its public search engine delivers fresh results fast, and it also serves as a showcase and testing ground for OEM partners.

To quote Dr. Rob Rubin, FAST general manager of the Internet Business Unit, “The same search technology developed by FAST to power Internet portals like Lycos and T-Online is used to provide ASP-based hosted search solutions for major corporations like IBM. We distribute our technology in shrink-wrapped packages and OEM for companies like Reuters and TIBCO. We also supply major e-commerce vendors like eBay and BroadVision.”

Rubin went on to explain that FAST has modest financial goals for AllTheWeb because it’s used not only to deliver results to a sophisticated user base, but also to test FAST’s technology. “We’re looking to cover costs, not drive profits – that’s for our OEM partners to do with their destination sites,” Rubin said

FAST did well financially, reporting a Q3 2001 revenue of $9.6 million, an increase of over 10 percent compared to the $8.7 million reported in Q2. Such performance doesn’t happen without some frugality. In 2001, FAST trimmed its overhead and narrowed its focus toward revenue-producing product suites such as FAST Web Search and FAST Data Search. By year-end, its workforce had shrunk from 230 to 170, and management resources became concentrated in Oslo, Norway, and Boston due to downsizing in the U.S., Norway, and the U.K.

Monetizing the Internet Business Unit

FAST consists of two major business units. The Internet Business Unit (AllTheWeb), competes with search engines like Google, Inktomi, and AltaVista. The Enterprise Business Unit, which powers Internet portals and provides OEM solutions to major corporations, competes with firms like Google, Inktomi, Autonomy, and Verity. In Q3 last year, the Enterprise Unit accounted for 60 percent of the revenue.

Plans are underway to further monetize the Internet unit by promoting AllTheWeb’s paid-inclusion PartnerSite service, which guarantees entry in the company’s databases for a per-URL fee (administered through partners like Terra Lycos). While this service does not  influence positioning or ranking, it gives visibility to many different file types and dynamic content.

PartnerSite includes a 24-hour refresh and distribution through all of FAST’s customers using the same technology (e.g., IBM). This option isn’t cost-effective for small sites, but it can be ideal for those who have time to monitor and rewrite each page every 24 hours to achieve high rankings (provided their SEO operations include staffing for this redundant monitoring and modification).

What’s been done to enhance AllTheWeb? “FAST just introduced its linguistics tool kit to automatically detect phrases and provide accurate search results” replied Rubin. I checked this out by entering “Who is Andrea Bocelli?” and “What is the weather in Rome, Italy?” Excellent references and fast!

The Deep Crawl

To index more of the Deep Web and run faster, FAST completely redesigned and relaunched its crawler technology last November. AllTheWeb can index multiple document types and crawls most dynamic content. For example, see the PDF documents indexed at Scirus or the video clips on Megasoccer.com.

How does dynamic indexing work? Stephen Baker, Director of Marketing and Business Development for the Internet Business Unit, answers that question for us. “FAST employs a real-time filter and fast indexing schedule for up-to-date information. Non-HTML files are indexed from HTTP and FTP servers using our state-of-the art crawling and indexing technologies. The index supports over 200 file types, including PDF, Real Media, JPEG’s, and a variety of executables. In addition, we have the ability to work with Web sites and content owners to incorporate database-driven content in the Web Search index through our FAST PartnerSite system.”

Companies with content in multiple formats can have their content indexed through FAST’s PartnerSite inclusion program. Firms like IBM have the option to decide which documents on their public Web sites they want to make available through the index. Note that many different document types are already available by FTP search on AllTheWeb.

FAST also created a real-time cluster in its data center to support news crawling, which is based on the same technology used to power Reuters. FAST uses this technology to continuously crawl 3,000 premier news sites, retaining the index for five days and continually refreshing it every two hours. The index is updated in sub-second latencies as new documents are crawled.

Tips for Top Listings

FAST suggests that webmasters focus on the basics, with emphasis on leading-paragraph content. “Clean design, good title text, clear descriptions, and good linkage from appropriate reference points within the site will help improve your rankings,” said Rubin. “Always focus on improving your site content.”

“The FAST robot starts off by checking Open Directory Project (ODP) descriptions. If none are found, it will look at Meta descriptions. Again, if none are found, it will examine the first 250 characters of content. Linguistic analysis of keyword frequency may reveal suspicious patterns, such as too many adjectives or the overuse of keywords. Such documents are flagged as spam,” he continued. I might add that FAST won’t tolerate cloaking or spam and fights vigorously to eradicate both, which includes getting offenders black listed.

Changing User Behavior

The percentage of users finding Web sites through search engines has fluctuated over the years (86 percent according to the GVU User Surveys and 42 percent or lower in other surveys). It’s a given that most people go to search engines seeking information. Rubin provided information from a Jupiter report (August 2001) in which Rob Leathern stated that (1) experienced users most often use search engines for needed data, and (2) about 30 percent of e-commerce purchases start at a general-purpose search engine. “FAST knows from its log investigations that people are launching their searches and navigating more using a public search engine,” added Rubin.

What’s in Store for the Future?

I think we’ve all noticed the trend toward categorization these days. Rubin predicts more navigation features in future search engines, much like ”FAST Topics” (listings shown at the top of the page to help users further refine searches within specific categories). “Search companies will concentrate on better intelligence, leading users to the data they seek. This includes categorization and phrase matching with larger phrase dictionaries” said Rubin.

Because of the continuing demand to access more Web content, Rubin believes we’ll see more specialized catalogs such as those created for science data at Scirus and for soccer information at Megasoccer.

Predicting a period of great innovation, Rubin says FAST is set to “attack the four pillars of search: relevancy, freshness, size, and speed.” Looks to me like they’ve got a good head start.

Five Ways To Improve Search Engine Rankings

Search engine visibility is so important to eBusiness success that building a Web site without optimizing for search engines is a wasted opportunity. Search engine traffic is highly targeted, providing an excellent ROI. Yet many eBusinesses don’t make search engine optimization (SEO) a priority.

Quality Traffic: Potential prospects are actively looking for your offerings on search portals so they’re predisposed to your marketing message. Companies can market through their Web site without making house calls.

• Software company Welocalize.com drives 90% of its site traffic through search engines, landing customers like Cisco, Xerox and MCI WorldCom. CEO Smith Yewell said he sold six figure jobs to those clients without ever having seen them face-to-face.

Low Cost: Search engine optimization is cost-effective. A key-phrase banner ad campaign runs $2,500 to $35,000 per month depending on the number of key phrases. A professional SEO campaign with an equivalent number of key phrases runs $1,000 to $10,000 per month.

Branding: An NPD Group study of recall for search engine text links and banner ad copy reported that search engines outperformed banner ads by three to one. More than twice as many respondents recognized companies listed in the top three search positions than those recognizing companies featured in banners ads.

Conversion Rates: Case studies based on client server file logs have shown that search engine traffic outperformed banner ads and email marketing campaigns by producing more purchases, form or catalog requests, newsletter subscribes, and software downloads.

Five Strategies for Higher Search Engine Rankings

Make your eBusiness Web site search engine friendly by focusing on five strategies to improve search engine rankings: keyword selection, Web page optimization, correct site submission, monthly monitoring, and improved link popularity.

1. Select Your Best Keyword Phrases

The most important thing you can do is to carefully research your keywords, identifying the phrases most likely used to find your Web site. Query employees, customers, vendors, family and friends. Look at your web logs to identify terms actually used to find your site. Spend some time with keyword popularity tools such as Wordtracker or WordSpot to finalize your best key phrases.

Prioritize your list and try a search for the key phrases generated. Search each phrase, noting how many similar documents are found and for which competitors. Scrutinize for relevancy and then optimize your pages only for your most relevant keyword phrases. Use two to three key phrases on the home page and two key phrases on each inner page.

2. Optimize Your Site for Customers and Search EnginesFirst, it’s necessary to write the best marketing copy for your customers. Use two or three of your strategic keywords at the top of each page, in your headline, in the first paragraph, and throughout the copy on your homepage. Then optimize each inner page the same way using two different keywords per page.

   Next, optimize for search engines by installing the proper “hidden text” in HTML program language that lies beneath the marketing copy visible on the Web. This helps search engine robots crawl and index your site.

   • Install proper title head tags.
   • Create proper keyword and description meta tags.
   • Create proper image ALT tags.
   • If using a <noframes> tag, insert keywords, descriptions, meta tags and content within the <noframes> tag.
   • If using JavaScript links or image map, add text links if possible.
   • Check the search engine compatibility of dynamically generated pages (.cgi, .asp, .cf, .php, etc.).
3. Submit Correctly to Major Engines and DirectoriesEach search engine and directory has specific criteria for accepting a request to be listed in its database. For best results, hand submit all URL’s. The order of submission matters.

   Manual Submissions: Most engines and directories post their “Add URL” instructions and links from the home page. Follow instructions carefully, and create bookmarks for future use.

   Dynamic Sites: Alta Vista, Inktomi and FAST now crawl dynamic sites. For crawlers that don’t pick up dynamic sites, you have two options: optimize and submit several static pages, or rewrite your dynamic URLs in a syntax that search engines can crawl (i.e., without the question mark (?) in the URL).

   Framed Sites: Google and Inktomi now index framed sites. For other crawlers, use the <noframes> tag or add meta content and a title to the frameset.

   Directories: Directories require hand submission. Carefully select your best category for proper ranking..

   Registration Confirmation: After submitting your pages, you must confirm registration. Each engine and directory has a timeframe for indexing your site which can take from two weeks to several months.

   Query the engine after the proper timeframe to confirm submission was successful. If you’re not listed after the indexing period, you must re-submit. Only re-submit after confirming you’re not registered as there are penalties for over-submitting.

4. Monitor and Maintain Your ListingsTo view and monitor your page submission results, run a keyword position report with TopDog or WebPosition SEO software. However, be careful… attempting to submit a lot of pages on one domain name, or submitting multiple domain names from the same IP address, can cause poor results.
5. Reviewing Results: After monitoring your pages across engines and directories, you’ll be auditing the results, which may call for:
   1. Re-submit (URL not found)
   2. Re-optimize (URL not found in top 30)
   3. No action required (URL found in top 30)

   When using software for monitoring, auditing, and reporting, these tasks should not be performed more than once or twice per month to avoid poor results. Only judicious use of monitoring and reporting techniques yields favorable results..

   Good Rankings Can Change: Because of the dynamic nature of search engines, optimization is a recurring process that doesn’t end once you achieve favorable rankings. Rankings can change from month to month — that’s why you need maintenance, recurring verification, and re-submission.

6. Improve Link PopularityMost search engine algorithms now use link popularity for relevancy, resulting in higher rankings. The rationale is that if your site has links from other popular sites, it must contain unique, relevant content. Following are a few strategies for improving link popularity.
   • Get Listed in Directories: Directory listings result in link popularity because they use human editors who only accept sites with unique, relevant content. They also seed many other databases.
   • Ask Popular Sites to Link to You: Search for your top key phrases, then review the sites that come up. Visit each site and ask those that are non-competitive to exchange links with your site, giving them reasons why your site is valuable and suggesting the page where a link to you might be advantageous. Links from these sites are valuable because their high rankings will improve your link popularity (and increase traffic if users click your link).
   • Modify Your Site to Promote Linkability: Provide quality informational resources and outgoing links on your site, such as links to search engines, news portals and industry professional organizations.
   • Start a Referral Program: A good way to increase links (and traffic) is to offer a referral or affiliate program. This must be done with proper content and not just a page of affiliate links. It takes time, research and planning but is an excellent source of quality links.
   • Put Content on Your Site: Write articles in your area of expertise and post them on your site. Submit articles to other Web publications and ezines.
   • More Information: Webmaster World’s Reciprocal Linkage Topics.

Mastering the above process can result in seeding search engines and directories to deliver qualified visitors deep into your preferred content areas. Targeted visitors will land on your product and service pages or commerce areas. This highly-qualified traffic brings excellent leads to convert into sales.

Should eBusiness Owners Optimize In-House or Outsource?

SEO is frequently handled in-house but this may not be your best option because most Web sites are not optimized properly despite in-house efforts. This isn’t surprising because SEO requires a considerable investment in personnel and technology to do the job right. CyberAtlas reported that eBusiness spends less than 1 percent of its marketing budget on SEO, even though this marketing strategy provides excellent ROI. This suggests that eBusiness could pay for the cost of outsourcing from the increased sales and profits resulting from professional search engine optimization.