Image: THOR/Flickr

Google’s site-blocking tool was originally aimed at “content farm spam,” but Google hasn’t done much with it of late, and it even stopped working for a while, despite being available via a link from your profile.

Now the service is officially gone, replaced by a Chrome add-on that does nearly the same thing. Unfortunately that means the ability to ban sites from Google’s search results is now limited to those using Google’s Chrome web browser. For more on the Chrome add-on see our earlier review.

The bad news about the Chrome extension is that it’s client-side filtering, not server-side. That means that if Google returns results from domains you’ve blocked those results are simply hidden (sometimes there’s even a brief flash of the blocked results).

That means you’ll end up with fewer search results than you would with the server-side solution, which filtered out your blocked domains before the results were sent. For example, if there are ten results on the first page and three are from domains you’ve blocked, using the add-on method you’ll only see seven results, whereas the server-side method would have fetched the next three results to show a total of ten.

If you used the account-based version of the blocking tool, you can head over to your account and grab the list of sites you had blocked. Just add those sites to the Chrome extension and you’ll be back up and running in no time, with not an Experts-Exchange, Quora or W3Schools link to be seen (or whatever you consider search results spam).

Home Page Photo: Carlos Luna / Flickr

GitHub’s Octobi Wan Catnobi. Image: GitHub

To help you find exactly the wheels your project needs, code hosting giant GitHub has announced a new, much more powerful search tool that peers inside GitHub repositories and offers dozens of filters to help you discover the code you need.

The new search further cements GitHub’s place as the go-to source not just for publishing, but also discovering, code on the web.

While GitHub’s new search lacks the web-wide reach of more general code search engines like Google’s once-mighty Code Search (now a hollow shell of its former self), it’s likely to return more useful results thanks to some nice extras like the ability to see recent activity and narrow results by the number of users, stars and forks.

GitHub’s advanced search page now supports operators like @username to limit results to just your repositories (or another user’s repos), code from only one repository (repo:name) or even code from a particular path within a repo. You can also limit by file extension, repo size, number of forks, number of stars, number of followers, number of repos and user location.

While the advanced operators make a quick way to search, there’s no need to memorize them all. The new advanced search form allows you to craft your query using multiple fields, while it displays the shorthand version at the top the page so you learn as you go.

Under the hood GitHub’s new search is powered by an ElasticSearch cluster which live-indexes your code as you push it to GitHub. The results you see will include any public repositories, as well as any private repositories that you have access to.

The GitHub blog also notes that, “to ensure better relevancy, we’re being conservative in what we add to the search index.” That means, for example, that forks will not be in search results (unless the fork has more stars than the parent repository). While that may mean you occasionally miss a bit of code, it goes a long way toward reducing a problem that plagues many other code search engines — the overwhelming amount of duplicate results.

GitHub’s more powerful search has turned up one unintended consequence — exposed data. It’s much easier to search for anything on the site, including, say, usernames and passwords. As it turns out many people seem to have everything from SSH keys to Gmail passwords stored in public GitHub repos. There’s a discussion about the issue over on Hacker News. The ability to find things like exposed passwords isn’t new, but the new search tool does make it easier than ever. Let this be a reminder of something that’s hopefully obvious to Webmonkey readers — never store passwords or private keys on a public site. And if you find someone doing that, do the right thing and let them know.

For more details on everything that’s new in GitHub’s search page, head on over to the GitHub blog.

Image: Don O’Brien/Flickr.

Google cautions that its new Disavow Links tool should be thought of as a last resort. It’s far better to get any spammy links actually removed from the web. In fact “the vast, vast majority of sites do not need to use this tool in any way,” writes Simon. But for situations where you can’t make the offending links go away — for example, with a client who might have made some bad SEO decisions in the past — Disavow Links offers a solution.

It’s worth noting though that Simon says that any links you disavow will be seen as “a strong suggestion rather than a directive — Google reserves the right to trust our own judgment for corner cases.”

Inbound links are perhaps the best known thing that Google uses to calculate PageRank and order search results. While PageRank is just one of more than 200 “signals” Google looks at to determine where your site will be in search results there’s no question that better inbound links mean your pages end up higher in search results.

There’s a flip side to inbound links though. If the wrong sort of sites point at your site it hurts your PageRank. If you’ve got inbound links from known paid link or other shady link-swapping schemes that violate Google’s guidelines, you can quickly find your site has disappeared from Google’s search index.

For more info on how the Disavow Links tool works, check out the video below from Google’s Matt Cutts. Also be sure to read through the FAQ over on the Google Webmaster Tools blog.

Everything in its right place. Image: Google.

If you’ve ever wished you could search all your mail and documents from the main Google.com search box, your day has arrived. Google is expanding its experimental integrated search features to make your mail and documents part of the Google.com search results.

Sign up for the trial and when you’re logged into your Google account you’ll be able to search Gmail and your Google Drive documents directly from the Google search page. Your mail and documents appear in a sidebar next to the usual results from around the web.

Google kicked off the Gmail search results on Google.com earlier this year with a limited “field trial.” Now, after what Google Software Engineer Bram Moolenaar (perhaps best known as the creator of Vim), calls “very positive feedback from those of you testing it out,” the company is expanding the universal search feature to a wider audience.

As Moolenaar writes, “when you search on Google.com, your results will include relevant information and messages from Gmail … and now — new in this field trial — also files, documents, spreadsheets and more from Google Drive.”

The updated trial also brings Google’s instant search results to Gmail. When you search in Gmail links to relevant email will pop up in the search bar as soon as you start typing — just like Google.com.

The new integrated search still isn’t the default behavior by any means, but it certainly looks like Google is moving in that direction. For now you’ll still need to sign up for the trial if you’d like to experiment with it. Note that the trial is only available in English and to those with @gmail.com addresses. (Google Apps accounts are out of luck for now.) If you opt in and decide you hate it, you can always go back to the sign up page and turn universal search off.

Searching for restaurants in the new Bing. Image: Screenshot/Webmonkey

Microsoft has taken the wraps off some new social features for its Bing search engine. The revamped Bing now uses a three-column layout and pulls in additional search results from your friends on Facebook.

A preview of the new version of Bing has been available for some time, but as of today the new layout and features are now available to everyone in the United States. So far there’s no word on when the same features might come to international users.

If social search features leave you feeling cold, fear not, the new Bing doesn’t follow Google’s lead and integrate social results right into the main search results. Instead Bing has completely overhauled its look and now sports a three-column layout. To go along with the new layout Bing has cut back on the chrome and other UI elements so even though there’s quite a bit more info on the page, it still feels relatively uncluttered.

The new look pairs the traditional search results — still on the left side of the page — with two new columns — a so-called snapshot column in the middle, and the new social results on the far right.

The most intriguing of the three is the snapshot info, which varies according to your search. The basic idea is to add extra, relevant information and services alongside your search results. For example, if you search for a restaurant Bing might provide a map or some reviews. Other searches will pull in relevant info for that search, with Bing doing a reasonable — though far from perfect — job of guessing what you might want to see.

The new social results require you to connect your Facebook account to Bing and will pull any relevant content from your friends into the far right column on Bing’s search page. There’s a new “Friends who might know” heading, where you’ll find a list of Facebook friends with any activity related to your search terms (including tagged photos). Unfortunately, in my testing not many queries brought up anything relevant from Facebook friends (which could say more about my friends than the new Bing, YMMV).

For more info on everything that’s new in the social revamp, be sure to read through the earlier announcement and check out the overview video from Microsoft Bing Director Stefan Weitz:

Where is that needle? Photo: Perry McKenna/Flickr.

Million Short seems like a terrible idea. Why would you want to remove the top sites on the web from your search results? In most cases you wouldn’t, but what Million Short offers is a chance to discover sites that just don’t make it to the top of the results from more popular search engines like Google, Bing or even DuckDuckGo.

It could be that these missing sites are just small, or perhaps they don’t use cutthroat SEO tactics to compete for popular terms, or maybe they just cover topics so niche they’re unlikely to rise to the top of any but the most targeted of searches. It could also be that they’re content farms and other worthless pages. Whatever the case, skimming the top million sites off the web just might open your eyes to how narrow your filters (and Google’s) have made your results, and how that’s both good and bad.

As Million Short notes, popularity is not an inverse corollary to quality, but when the same popular sites show up over and over in your results you are inevitably missing out on something. And that’s what Million Short wants to show you.

It’s important to realize that Million Short is removing the top websites not just the top search results for individual queries. It’s also worth noting that Million Short doesn’t disclose where its search results are from, nor how it calculates the top sites. [Update: Sanjay Arora, founder of Exponential Labs, tells Webmonkey that Million Short is using "the Bing API... augmented with some of our own data" for search results. What constitutes a "top site" in Million Short is determined by Alexa and Million Short's own crawl data.]

Most of the time, narrowing search results down to trusted, well-known sites like Google, Bing and other search engines do is a good thing. To see why just plug a few programming queries in Million Short and you’ll quickly realize just how helpful Stack Overflow — well inside the web’s top 1 million sites — has become. At the same time you might discover some unknown blog that will never make the top results in Google and happens to have the answer to exactly your problem. Is that better than the same answer from Stack Overflow? That’s up to you.

Million Short does offer some customization options you can use to both cut out the top sites and keep the handful you don’t want to be without. Additionally you can change the limit from the top million to the top 100,000, 10,000, 1,000 or 100 sites. If you decide you love it there is a search engine plugin that will work in Firefox, Chrome and Internet Explorer.

Perhaps the better way to think of Million Short is not so much a search engine, but a discovery engine. Million Short’s strength is not going to be answering the specific kind of queries that Google is forever optimizing its index to handle, but to discover less well-known sites and explore the more remote corners of the web that might be lost in other search indexes.

Searching XKCD. Image: DuckDuckGo.

DuckDuckGo already offers quite a few of what the company refers to as goodies — quick answers and clever shortcuts for common search tasks. For example type “1 + 1″ in DuckDuckGo’s search field and you’ll get “1 + 1 = 2″ in addition to your search results. Other goodies include time based queries, unit conversions, facts (for example the weight of a penny and of course DuckDuckGo’s “!bang” syntax for searching specific websites). For a full list of all the built in goodies, see DuckDuckGo.

DuckDuckHack takes the goodies concept and crowdsources it. Now anyone can write a custom plugin for DuckDuckGo and everyone can take advantage of it. There is already a number of cool plugins available, including the XKCD comic search pictured above, as well as more generally useful tools like lyrics search, a Twitter username search or an em-to-pixel converter for web developers.

Anyone can build DuckDuckHack plugins and the idea is for developers to build tools that they personally care about — scratch your own itch and pass it along so everyone benefits. For more info on how to write a plugin (DuckDuckHack plugins are written in a variety of languages depending on the type of search), check out the tutorial and guidelines for developers. DuckDuckHack takes requests as well. If you’ve got an idea for a search shortcut, you can let developers know.

While the new DuckDuckHack plugins idea makes DuckDuckGo an even more compelling Google alternative, some of DuckDuckHack is still a bit rough around the edges and it’s unclear what DuckDuckGo plans to do if two developers build two plugins that conflict by both responding to the same keyword trigger.

Still, DuckDuckHack is only a day old and already it’s added several useful new tools to DuckDuckGo. To learn more about the new plugin search features already a part of DuckDuckGo, visit the DuckDuckGo goodies page.

Shortly after Google launched Search plus Your World earlier this month, critics accused the company of favoring its own nascent social network over the much richer results on others, like Twitter or Facebook. As Wired’s Steven Levy quipped, “there’s too much Plus and not enough of Our World, which has oodles of content on other social networks.”

Now developers at Twitter, Facebook and MySpace have put together a demonstration of just how much relevancy Google sacrifices in order to push Google+. The demo, which uses only Google’s own results, shows, among other questionable results, how Google routinely ignores more relevant Twitter pages to show off seldom-used Google+ profiles. To see it in action, head on over to the new Focus on the User website.

If you decide you prefer the often more relevant results from the Focus on the User experiment there’s a bookmarklet available, cheekily entitled “don’t be evil.” Just drag the bookmarklet into your web browser’s bookmarks bar and then click it whenever you want to see more than just Google+ results in Google’s search results.

The developers behind Focus on the User do work for Google+ rivals, but that doesn’t change the results of the experiment which speak for themselves. The developers also point out that their tool relies entirely on Google’s own data to rank social search results. Here’s their description of how the don’t be evil tool works:

the tool identifies the social profiles within the first ten pages of Google results (top 100 results). The ones Google ranks highest — whether they are from Flickr, Twitter, Facebook, LinkedIn, MySpace, Quora, Tumblr, Foursquare, Crunchbase, FriendFeed, Stack Overflow, Github or Google+ — replace the previous results that could only be from Google+.

In other words the bookmarklet largely returns Google to its previous state, before the Search Plus Your World Update. If you’d like to know more about how the bookmarklet works or see some examples and situations in which the emphasis on Google+ social results actually degrades the quality of search results be sure to check out the video below.

Photo: Rene Tillmann/AP

Google has tweaked its search algorithm to punish websites with excessive advertising “above-the-fold,” that is, websites that stack the top of the page with nothing but advertisements.

According to Google, “rather than scrolling down the page past a slew of ads, users want to see content right away.” To help users get to that content, Google may drop ad-heavy websites from its search results.

Google says that the change will only affect about one in 100 searches, and emphasizes that websites using what Google’s Distinguished Engineer and SEO guru Matt Cutts calls “ads above-the-fold to a normal degree” will not be affected.

Instead the change is designed to punish sites that “go much further to load the top of the page with ads to an excessive degree or that make it hard to find the actual original content on the page.” In other words, if a site is so packed with ads that people can’t find what they’re looking for then Google isn’t going to send them to that site anymore.

While the distinction seems clear at first glance, digging deeper reveals some potential confusion for webmasters — for example, what role does screen size play? On a netbook, for instance, Google’s own search results page is almost entirely taken over by advertisements, not the actual search results (i.e., the content).

Google on a netbook screen: Ads are in red, search results in green

At small screen resolutions, Google’s own search results page is one of the worst offenders when it comes to advertising clutter obscuring content. That seeming hypocrisy may leave some webmasters wondering what constitutes “a normal degree of ads” and how screen size affects what is defined as “normal.” Sticking simply with what Google has written about the change, copying Google’s search results page is probably not a good idea in this case.

Cutts does encourage webmasters view their websites at different screen resolutions, suggesting that screen size does play a role, but unfortunately he doesn’t offer any details about what that role is or how it affects the algorithm’s new layout ranking scheme.