New Bing Maps Beta with embedded data layers from Twitter and other social feeds, click to enlarge screenshot

There’s a lot of information on the Internet already. Every day, more is added – a lot more. And while there are a concomitant number of new analytic or sense-making tools on the web, they butt up against the fact that the data – the all-important data – is held in multiple places, formats, and platforms.

How are we going to deal with all this? One approach is almost mechanical: ensuring that datasets can be accessed commonly, as in our new Microsoft Dallas platform associated with the Windows Azure cloud platform.  In the government realm, the anticipated reliance on “government-as-a-platform” (a meme popularized by Tim O’Reilly) holds promise in allowing somewhat aggregated datasets, openly accessible.

Another approach is at the far end of utility, on the post-analytic end with better search and decision engines. That’s why we’ve just upgraded Bing with some phenomenal features (see “Bing Maps Spawn the ‘Decade of the Interface?’ IDC thinks Bing Maps is on the start of big advances in Web user interfaces“).  In a neat demonstration of the value of competition, by the way, Google just followed suit a week later with some strong new features as well.

But an additional, and powerful, approach in the middle of the data value-chain will be to use semantic techniques on all of the data itself. A couple of weeks ago we unveiled the Microsoft Semantic Engine, which brings search, structured query, and analytics together in a simple, unified Search-Discover-Organize usage model.

Having all forms of digital information on a single platform allows people to blend unstructured and structured content and to drive insight and decision making

 Microsoft Semantic Engine provides a combination of technologies to form a contextual understanding of all digital content

MSE has been designed to allow unified access to structured and unstructured enterprise data through easy-to-use analytical tools. Microsoft might build some of those tools, and we’ll certainly integrate MSE capabilities into our own “information-worker” software, just as it is going to be integrated into Windows 7 desktop search and other apps. But the platform is meant to be easily available for use by the world of developers and partners, so that their independent innovations can take advantage of the semantic engine.

MSE Architecture, showing array of pluggable services including inference engines, ontology and taxonomy managers, image and text processors etc. Click to expand screenshot.

Therefore we held the unveiling at our Professional Developers Conference in Los Angeles. While MSE is not available for use today, it’s right around the corner, so we are busily providing information to the developers and partners whom we hope will be using the engine.

You can get the same experience that PDC attendees had, by watching the video of the presentation and demo here, or downloading the slides which accompanied the roll-out and live demo. The video itself is about 45 minutes long and features talk and demoing by Naveen Garg and Duncan Davenport of the MSE team.

I first met Naveen two years ago, not long after joining Microsoft, when I began hunting for smart people working on semantic projects like one I had just left in the intelligence community – our “Alien” project (All-Source Intelligence Environment).  I’ll point out that at 10:30 into the video, Naveen speaks for a moment or so about the interest within government circles in our semantic approach; he is referring to discussions we have had with visitors I’ve brought to Redmond from Washington DC for a look at the engine in its early lab days. They enjoyed meeting Naveen and his team, and I have to say they’ve cut us a bit of slack as the MSE team hunkered down and focused quietly on their work.

MSE is not yet a “product” for sale by Microsoft, and may not become one per se, at least by itself. We’re still going through the process of figuring out exactly how it will be packaged. But the rationale for surfacing it now is that we want to encourage developers who are already working in knowledge-rich corporate environments or other information-centric arenas, to be prepared for the ability to take control over structured and unstructured content, in a relational way that will provide contextualized, personalized insight into massive data for users – whether they’re web surfers or enterprise users.

Here are some of the scenarios we believe are relevant for using MSE:

• Search and Collaboration | Personalized search, discovery and organization
• Legal | Precedent and subject based search over large scale textual corpuses
• Life Sciences | Systems biology with large volume data correlation and search
• Government Services | Intelligence, real-time analytics, visualization, clustering
• Social Networking | Social graph relevance mining, ranking criteria

MSE Demo, with screenshots of Search, Discover and Organize features, exposed via UX gallery using fully-documented managed APIs. Click to expand screenshot.

Just a couple of highlights in the video to watch for: At 36:15 you’ll see an integrated MSE semantic search within Windows 7 Desktop Search, which shows the underlying SQL Server search going on, with all semantic properties exposed and analyzed automatically for relevance and semantic relationships within the Desktop Search UI.

And at 39:30, there’s a quick 3-minute technical discussion, describing how we’ve integrated the Semantic Engine Database, the System Integration Fabric, the API Layer, and the entity processors and classifiers against different data types.

If you want a bit more technical take on the presentation, check out this blog from a SQL Server business-intelligence (BI) developer, who is excited about the possibilities he sees from using MSE.

I’m excited too – I’m a big believer in Our Glorious Semantic Future and I believe that Microsoft now has a firm footing from which to build great capabilities. If you want to stay current on our progress and plans with MSE, feel free to drop me a line.

Share this post on Twitter

Email this post to a friend

Have been using Google Desktop for finding files quickly. most often, I was just looking for filenames. Cleaning the system recently, saw that the search index is a massive 11.0 GB! Looked for Open Source alternatives.
Installed and tried Locate32. Not a shiny interface, does not search emails but for filenames its fast! and the index is around 20 MB!
Locate32 from now on!

Powered by ScribeFire.

My paper ‘Retrieval Experiments in Pseudo-desktop Collections’ (co-authored with my advisor Bruce Croft) will be presented in CIKM2009. It is about a new model of desktop search research, where we introduced ‘pseudo-desktop’ — a simulated desktop collection composed of automatically gathered documents and generated queries. The method to validate generated collection is suggested as well.

For retrieval model perspective, we saw desktop search as a known-item search task over semistructured document collection, since people usually find what they already know of and each document in desktop has metadata. For instance, e-mail has sender and receiver fields in addition to usual title and content fields. We presented an improved retrieval model based on PRM-S, which was introduced in my previous work.

I believe that the significane of this work is threefold. First off, it is an effort to bring more scientific effort to desktop search (or searching personal information in general). People have studied desktop search for a long time yet they mostly built their own systems and reported the result of user-study, which lacks reproducibility. Pseudo-desktop can be a sharable data collection which address this problem, by which new researchers can test their algorithm against state-of-the-art baselines without building yet another desktop search engine.

Also, the experimental result shows the value of simulation as a method in IR research. Simulated query is not only free to get but also provides total control over the parameters. In our experiment, using algorithmically-generated queries many different characteristics, we could find insights over the performance of tested retrieval methods. Another paper in SIGIR2009 also demonstrated this value of simulated queries.

Lastly, PRM-S — a novel retrieval model based on the mapping between query-word and document structure — was found to be useful in a noisy settings like e-mails (e.g. many word-overlap between document fields) as well as the collection of clean structure like movie database.

Recently, I’ve been working on the development of LiFiDeA — a prototype PIM system, by which I plan to compare the experimental results of pseudo-desktop and real-desktop collections. I believe that this will provide ultimate validation for our approach here.

Desktop Search:

Windows Search 4

Google Desktop – Read More

Um Dateien auf dem eigenen PC zu finden, kann man auf zahlreiche schnellere und komfortablere Alternativen zur Windows Standard-Suche zurückgreifen. Die Desktop-Suchen lassen sich dabei nach Suchmethode grob in zwei Kategorien aufteilen. Einerseits gibt es jene, die zuerst einen Index bilden indem sie die Festplatte scannen, und später nur die eigene Datenbank durchsuchen. Das hat den Vorteil, dass die Ergebnisse beim Suchen sehr schnell angezeigt werden – normalerweise schon beim Eintippen. Nachteil: der Index ist immer nur kurz aktuell, neue Dateien werden erst einmal nicht gefunden und müssen erst indexiert werden. Und je nachdem, wie effizient das Programm scannt und wie tiefgehend die Suchkriterien sein sollen (z.B. wenn man auch in Dokumenten suchen möchte), kann das Bilden der Datenbank relativ lange dauern. Für aktuelle Ergebnisse muss das Tool am Besten immer laufen.

Copernic Desktop Search

Zwei bekanntere Vertreter dieser Kategorie sind Copernic Desktop Search und Superior Search. Beide erlauben auch das Suchen in Dokumenten und Emails. Superior Search findet außerdem auch ähnlich geschriebene Dateien – gut, wenn man sich vertippt oder den Namen nur ungefähr kennt. Wer auf das Durchsuchen von Dokument-Inhalten verzichten kann, hat mit Everything eine extrem schnelle Alternative. Auch Locate32 punktet mit Geschwindigkeit. Beide liefern schon beim Eintippen Resultate und scannen ganze Festplatten in Rekordzeit. Everything indexiert zudem automatisch im Hintergrund – der User braucht sich um gar nichts kümmern.

Superior Search

Die andere Kategorie bilden die Tools, die erst suchen, nachdem der User seine Anfrage eingegeben hat. Das dauert dann länger, Ergebnisse kommen nicht in Echtzeit. Aber diese Desktop-Suchen finden immer alle Dateien, auch solche, die man gerade erst gespeichert oder verschoben hat. Und natürlich dauert so eine Suche nicht so lange wie das Indexieren aller Dateien auf dem PC.

SearchMyFiles

Auf diese Art arbeiten z.B. Super Finder XT und SearchMyFiles. Letzteres kommt ohne viel optischen Schnickschnack aus. Statt dessen konzentriert sich das Tool auf das Wesentliche: die Suche bietet viele Optionen und ist relativ flink. Dabei verbraucht das Programm kaum Systemressourcen.

Super Finder XT hingegen sieht einfach ein bisschen schicker aus. Bei der Bedienung orientiert sich die Suche am Ribbon-Design von Microsofts Office 2007. Außerdem ist die Suche in Dokumenten möglich.

Super Finder XT

Zwar bieten so ziemlich alle großen Suchmaschine-Betreiber auch eigene Desktop-Suchen an, doch denen sollte man mit gesunder Skepsis begegnen. Mißtrauen ist vor allem bei im Browser laufenden Anwendungen geboten, die die Verschmelzung von Web- und Desktop-Suche versprechen. Denn wer möchte ernsthaft den gesamten Inhalt seines PCs scannen, auswerten und die Ergebnisse über das Internet an irgendeine Firma übertragen lassen? Aus Sicherheitsgründen wäre es deshalb sinnvoll, einer Desktop-Suche keine Verbindung in die Außenwelt zu gestatten.

Soviel zur Theorie, in der Praxis entscheidet wahrscheinlich Design und Funktionalität, welches Tool man letztendlich bevorzugt. Die vorgestellten Programme haben alle ihre speziellen Vorteile – am Besten man probiert sie einfach mal aus.

Wer seinen Such-Favoriten gefunden hat, kann noch einen Schritt weiter gehen und die Schaltfläche “Suchen” im Start-Menü mit der neuen Desktop-Suche verbinden.

Viele User nutzen nicht die Windows-eigene Suchfunktion, sondern eine alternative Desktop-Suche wie z.B. Everything oder Copernic. Der Eintrag “Suchen” im Startmenü verweist aber weiter auf die Windows-Suche. Mit RerouteXPSearch kann man das ändern. Dazu startet man das Programm, gibt den Pfad zur neuen Desktop-Suche an und klickt “Apply”. Das Beispiel unten zeigt den Pfad für Everything. Falls das Ganze wider erwarten nicht funktioniert, startet man das Programm nochmal und klickt “Restore Defaults”.

RerouteXPSearch gibt es hier zum Download (Runtersrollen bis “Attachements”)

Do you like the old Search dialog in Windows – the Search Companion?  I can’t stand the new one, it brings back too much.  If you want the old Search Companion to appear when you start a Windows Search (by click Start – Search on the screen or by pressing the Start + F keys), it’s easy to get it back. 

1. Open your Registry by clicking Start – Run and typing Regedt32 in the dialog box
2. Go to HKEY_CURRENT_USER\Software\Microsoft\Windows Desktop Search\DS
3. Change the key ShowStartSearchBand from a ‘1′ to a ‘0′ (zero)
4. Close the Registry

Next time you open your search, you’ll be back to the good search box!  This tip works on Windows XP.

For a long time I was a complete mac fan and I started making my windows xp look like a mac. I even pasted stickers and tattoos on my laptop to make my HP laptop with windows xp look like a mack book. After playing around with it a lot, I decided to move on to Ubuntu for two main reasons. One because it was free and the other reason was it looked pretty much similar to Mac and I cannot afford for a mac. It worked well for long time, but my work nature demanded and forced me to use windows. I then went back to windows and started finding tools that would make my life easier with windows and slowly I started liking windows. One of those features that made me like windows was the instant search tools that I found. One was Windows search(WS), that searches your hard disk for files instantly. That looked like a spotlight(instant search tool for mac osX) for windows xp, though WS does not perform all operations that the spotlight does, does not look like spotlight as well. But it is super cool and helpful tool.

Two features due to which spotlight stands above WS is

1. Spotlight can be found on any explorer window and is available on almost all application that is search enabled. One can search for a file, placed in any folder using the spotlight from any explorer window. That’s a stroke of a genius. (this is now available in Windows Vista)

2. Spotlight gives search option even on the insert file window (the window that comes on when you click on the attach file button on any mail client or on any browse button). Another brilliant stroke.

I have been looking for a tool for xp that has the above features but till now I have been unsuccessful. I would love to have something like that.

I came across another tool called QT tab. This gives an instant search options on all explorer windows, however the search is limited to the files and folders in that particular window. This tool also gives you an option of tabbed browsing in the explorer window.

There are so many other tools that gives these kind of search facility but most of them are not stable softwares. They crash every now and then. The front runners could be

Google desktop search

Copernic search

Windows search

However one draw-back on all these tools when compared with the spotlight is that, all the above application eats up a lot of memory. At the end of the day, though I have told earlier that I have started liking windows and linux, I am still a huge fan of mac. Not only for this reason, there are lot more which I will keep adding on in different posts.

If you’ve got more to say about your search-tool-of-choice, let’s hear all about it in the comments.

I first used Search in the mid 80’s (Digital Equipment Search) It became a most useful tool for debugging and enhancing existing systems. Years later when I got my first PC I decided to create a Desktop Search. Just like the one DEC had. That was 1999.  Features got added (pictures, Video and Music) options and much more. Everything a programmer with nearly 40 years of experience would want. And it’s open source.

I’ll be showing how this program can make your computer life simple and fun.

Num artigo anterior expus, ainda que de forma muito leve, maneiras de lidar com a sobrecarga de informação que recebemos e consultamos diariamente.

Tony Karrer, no seu blogue eLearning Technology, publicou um artigo excelente sobre este tema. Em Better Memory, ele apresenta um conjunto de estratégias para “memorizar” páginas e informação que consulta amiúde e que tem interesse em preservar para consulta futura, recorrendo ao social bookmarking usando o Delicious, ao histórico da navegação através do Google History ou pesquisando no ambiente de trabalho com o Microsoft Desktop Search ou o Google Desktop Search.

Recomendo vivamente o artigo e a consulta regular do blogue.

I’ve two goals for this space. First, to help you be more productive, make more money, and enjoy more success. Second, to create a dialogue with you and other readers around how to compete and win in an increasingly connected world.   

Despite so much being written and talked about around the “unprecedented” challenges we face, I’m convinced there’s never been a better time to be in business than right now. Crises really do present opportunities – just not to everybody. As Abraham Lincoln said, “Good things come to those who wait, but only what’s left by those who hustle.”  

I’m optimistic for a simple reason: if knowledge is power, never in history have people had such ready access to it. Technology and the web give individuals today more power at their fingertips, and more reach, than even the largest Fortune 500 companies had just a generation ago.  

But that’s a meaningless advantage if you can’t capitalize on it.

That’s why, in the coming months, I’ll be outlining strategies, tactics, and tools you can employ to outsmart and outhustle your competition at every level, using resources you already own or can easily access — whether you work alone, manage a large team, or run a multi-million dollar enterprise.  

As with my speaking, consulting, coaching and other writing, the mission here is to cut through the chatter and outline concrete, specific, practical ways you can immediately boost your productivity and your bottom line.  

In a few days I’ll be talking about one of the easiest (and essential) ways you can begin saving time and effort — by inventorying and repurposing materials you’ve already created. You’ll be amazed how quickly you can throttle back on recreating versions of the same stuff from scratch over and over again.  

Meanwhile please feel welcome to share your successes, your suggestions, or your questions around topics you’d like to see covered here in the future.  

Thanks for visiting. 

‘Alles’ is geen bescheiden naam voor een programma. De maker van deze zoektool voor harde schijven rechtvaardigt zijn hoogmoed door ogenblikkelijk de inhoud van uw computer geordend weer te geven.

Zoekprogramma’s voor het bureaublad zijn meestal te verdelen in twee groepen: software die voortdurend uw bestanden indexeert (Copernic Desktop Search, Google Desktop) en rekenkracht opeist voor zijn snelheid, en sobere applicaties die minder luxe bieden maar weinig van uw systeem vragen (zoals Locate32 en DK Finder). Everything combineert de voordelen van beide methodes en verslaat alle concurrenten wanneer het op snelheid aankomt.

Everything is zo snel dat de resultaten al in het sobere hoofdvenster verschijnen terwijl u een zoekopdracht ingeeft. De interface gedraagt zich exact als een Windows Explorer-venster, zodat u meteen bestanden en mappen kunt openen, kopiëren of verwijderen. De zoektool geeft resultaten in real-time weer. Zo kunt u een bestand opzoeken en kopiëren, en na voltooing duikt het gekopieerde bestand meteen op in uw zoekresultaten.

Homepage link

Have you ever misplaced a file on your computer? Or maybe you’re looking for a file someone else created but don’t know where it is. Instead of wasting your time digging through folders and files you can get a software program to do the job for you.

There’s a few of them out there.

• Google Desktop

This one’s fun because you can customize it with gadgets like weather reports and news.

• Copernic Desktop

In addition to the free home version Copernic offers professional and corporate versions for a fee. Free downloads are available in 4 languages:

1. English
2. Spanish
3. French
4. German

• Yahoo! Desktop

If you’re a Yahoo! fan this desktop search tool is integrated with other Yahoo! features.

I’ve been thinking about whether I should install a desktop search facility on my machine. At the moment, it is fairly well-structured and after only a year I don’t have too much stuff on it. However, this is changing rapidly as my work levels ramp up, and I wonder if my taxonomy will be enough to remind me where everything is!

In the past, I’ve used Lookout (now part of Windows search), Google desktop, Copernic and X1 (now part of Yahoo desktop search I think). My problem with them is generally the same – they have a huge footprint on my memory (RAM) and really slow down my machine. Of thes I found that Copernic worked best i.e. found the most relevant results.

I’ve recently heard of another couple of apps called DK Finder and Locate32 which are meant to take up much less RAM. I’d be interested to see a decent list and comparison of the top apps in this area, enlarging and updating the one Pandia did some time ago. Anyone know whether this has been done (this year)? I found a matrix done by the Goebel group, but I would like more comparison on the features of each product, and of course – the amount of RAM taken up.

I think windows xp’s search assistant sucks, unlike vista’s.
I found that the windows desktop search from microsoft is cool it makes your desktop search like vista’s
It works fine with all xp editions.
website: WDS windows desktop search