Find that file quickly with the Dynamic Query tool, which:
- Searches through an index of files' contents as well as names
- Works as you type--no waiting!
- Is faster than browsing for files, especially in deep folder hierarchies
Setup
The Dynamic Query tool requires Windows 2000 or XP (or anything newer).-
In the "Computer Management" administrative tool (Control Panel: Administrative Tools: Computer Management)
- In the left panel, open "Services and Applications" and then "Services"
- Ensure that "Indexing Service" is started (and preferably set it to start when Windows boots)
- Now in the left panel, open "Indexing Service"
- Add a catalog named "documents"
- Add all of the folders that you want to search to the "documents" catalog
- Download, unzip, and run the Dynamic Query executable file
Theory
Overview
The system is designed as a replacement for the standard Windows Open file dialog. Instead of browsing to find a file, the user types in a search string. Traditional search systems require the user to enter a query, click a button, and then wait for the query to be processed. The results of a dynamic search are displayed as a query is typed, which makes the interface more interactive. The interface is clean and simple to eliminate the overhead of searching.
Technical
The tool was built on top of Microsoft Windows 2000/XP Indexing Service. The tool was built with Visual Basic. Indexing Service allows users to create catalogs. These catalogs can contain any number of directories; these directories and all of their subdirectories will be indexed by the service. The tool allows users to query a documents catalog. This catalog must currently be manually set up by the user but eventually should be automatically set up to include the Desktop and the My Documents folder.
Freeform Keyword Entry
To search for files, the user enters keywords into the search string in the text field at the top of the screen. The tool looks for files that include the keywords in their titles, file names, or contents. Every time the search string changes a new query is executed, and the results are displayed as a list in the bottom part of the window.
Technical
The software creates a query by splitting the search string into an array of keywords, applying a CONTAINS operator that searches for each keyword in the title, filename and contents of the files in the database, and joining these together with AND operators. For example, if the user types in, grant proposal, the software will return documents that have grant in at least one field and proposal in at least one field.
The query generator has been tweaked a bit to make it work better with dynamic queries. Since the last keyword in the search string is usually incomplete, the query looks for words that start with the last keyword (whereas all the other keywords are whole-word-only). If the user typed grant pro, the software would look for documents that match the whole word grant and the start of a word pro*.
The search currently matches keywords only in the title, file name and contents fields of the files in the database. It does not match results in author, file path, or any other field. This is an important design decision: the fields that are considered in a search uniquely identify files, while the fields that are not considered do not. A search for Administrator that included the author field could return a huge number of files unrelated to what the user is interested in. Similarly, a search that looked in the file path could return many files not directly related to the users query. For instance, the user may have an application stored in a folder on their desktop named Eclipse that stores a software application. If the user searches for Eclipse and the software matches the keyword in the path, it will return all of the applications data files—not what the user is looking for. While it would be useful to be able to query across all fields, letting queries span all fields by default could easily confuse the user. To remedy this confusion about files that satisfy the query in different fields, it could be valuable to build an interface similar to grouping in Microsoft Outlook that groups files based on which fields match the query. Then results that arent related to the users query but are displayed because of some obscure field can easily be ignored.
The system is reasonably quick for an index of 34,000 documents on a Pentium III 800MHz with 512Mb RAM. It currently has two modifications for speed: it doesnt start querying until the search string is 3 or more characters long (nearly all searches are longer than 3 characters), and it only shows the first 100 documents returned by a query (since the search is dynamic, its easier for the user to make the query more specific than to scroll down to see a specific document). Even so, the system could be quicker. When a user adds a character to a query, the system queries the whole document database, but it could just look for a subset of the documents it has already found (since wed just be making the search more specific).
Results
The results display looks like a Windows Explorer details view. It will offer the same right click context menu options as the standard Windows search results view [this has not been implemented yet]:
Windows Explorer Search Results View
Observations
The tool makes retrieving files that are located deep within a directory structure much quicker—its much faster to type a few keywords than to drill down to a file using Windows Explorer. Furthermore, if you know anything in the contents of a file that uniquely identifies it, you can easily find it—even without knowing its location. The tool flattens the directory structure so that the files representation is independent of its location.
The most pronounced difference between using this tool and a conventional non dynamic search tool is the sense of stability it provides. The user is in control at every step of the search process, the search process is quick, and he/she can see immediately if a mistake is made in entering the query. Furthermore, the interface is much less intimidating with its one freeform query box than the typical search interface that consists of rows and rows of boxes for obscure fields. The dynamic search process feels much more comfortable than Windows traditional Search for Files and Folders dialog.
Applications
The ideas of dynamic searches could be helpful when implemented in different contexts and for different applications. Some examples are listed below:
A hard drive cleanup utility could use dynamic sliders for age and file size to help users find large, old files that they no longer need. (this application is suggested in Browsing Unix Directories with Dynamic Queries: An Evaluation of Three Information Display Techniques)
A filename auto completion utility could take the place of the address bar auto completion feature in Windows that suggests options as a user types a path:
The current implementation in Windows simply lists the files and subfolders contained within the path that start with the letters that the user has already typed. A revamped version could run a deep search and show only recent files. A deep search would look for the keywords the user is typing in across all folders and subfolders, not just the current one. This search would match keywords in file names and paths. It could help the user cut through his/her folder hierarchy to quickly get to the file he/she is looking for. It could also show recently modified entries first, since its most probable that a user will be trying to open a document that is in a state of revision—one that has been modified recently.
An archive search utility could help users dig through troves of historical and autobiographical information for important information. Users may only remember one small feature about information they are looking for; the system should be able to retrieve information based on these random observations. I may remember that the book Im looking for is red, but forget the title—a good search utility will still help me find the book. The dynamic query tool presented in this paper is such a tool, for electronic files. It searches file contents as well as filename and title. If a user remembers just one unique phrase from the file he/she is looking for, the tool will be able to quickly retrieve it.
Dynamic search could also be applied to e-mail. In E-mail as Habitat, Ducheneaut and Bellotti find that few users report much use of the search feature of their e-mail client; on the contrary, however, almost all respondents say they frequently use the sort feature…The sort feature appears to be more popular than search because it can be used to more quickly specify search criteria (34). I think that a dynamic search tool would fare far better than the traditional search offered by the most e-mail applications, and would also beat out the use of sorting features for retrieval. Such a search could provide a text box for keywords that can match across common e-mail fields like the subject, to addresses, from address, and date.
Apple's OS X e-mail client provides a dynamic search utility, but since the software doesnt maintain a local index of the users inbox, the tool is extremely slow. Scopeware Vision, though it is not a dynamic search tool, duplicates a users inbox in the local filesystem and uses Indexing Service to produce an index. Using a similar method as Vision, my tool could be extended to provide dynamic searching across e-mail as well as local documents. A tool that searches across e-mail and files could be used to study the similarities and differences in retrieval patterns for the two forms.
Impact
It is interesting to consider the ways in which dynamic search could impact storage and retrieval processes. The following section looks into how search can add to or replace the retrieval process of browsing and the storage process of filing.
How can dynamic search coexist with browsing? The filename auto completion utility presents a way in which dynamic search can coexist with browsing—users can utilize either method, and both methods are integrated. Dynamic search could be used to find a specific file, and then browse could be used to find other files in the same folder.
In Browsing vs. Search: Can We Find a Synergy?, several researchers views on bringing together browsing and search are presented. Among the suggestions by George Furnas are Navigation by query, where the user essentially drills down to a specific piece of information by submitting a sequence of queries. After seeing the result of one query, the user constructs the next query to refine the search. The process is iterative, like browsing, but also direct, like search. Navigation by query is a generalization of the dynamic query system presented in this paper that may prove useful in other, unrelated applications.
Can dynamic search replace browsing? Is it possible to effectively navigate through a users documents using only a search tool and without browsing? It could be possible that dynamic search can take over from browsing as a retrieval process, but I would venture to say no. A user may not be able to remember anything about the file he/she is looking for, or may be storing it in a folder for reminding purposes. Searching for a file is a process of active retrieval—you need to know what you are looking for in order to use search. One way to test if dynamic search could replace browsing would be to replace the open file dialogs in all of a users applications with the dynamic search tool, and see how well they are able to bring up the files they are looking for.
How can search coexist with filing? Here we consider three types of storage systems:
Intentional hierarchies are set up by the user. Users manually add or remove items from folders or categories. Examples include the folders used in Windows Explorer and items in outliner software like Ecco.
Extensional hierarchies are automatically populated by software. In general, users create rules that define what should be stored where. Examples include virtual folders or e-mail folders whose contents are governed by rules (such as, file all messages from this e-mail list into this folder).
Presto presents a hybrid hierarchy system, where the software uses a rule to determine the contents of a folder, but users can also manually add or remove items (users have the final say).
The filing process is used with intentional hierarchies—users manually categorize items into folders and subfolders. Extensional hierarchies are essentially a collection of saved searches—items that match a certain query show up in a particular folder. Traditionally search is thought of as a retrieval process, but with extensional hierarchies it is seen that search can function as a storage process as well.
Prestos hybrid system presents a combination of filing and search that offers the benefits of both intentional and extensional hierarchies—the user gets the final say on what belongs in a folder or category, but the software does most of the work. Combining this type of hybrid system—which is usually coupled to a traditional search system—to a dynamic search would be an interesting direction to take this research. Traditional searches have a high overhead and are not as natural as dynamic searches; by meshing dynamic search with filing in a hybrid system I believe it is possible to create an efficient and natural storage solution.
Conclusion
We have explored one limited application of dynamic search and have found great potential—dynamic search is faster and more natural than traditional searches. The applications of dynamic search are far reaching: the system could be integrated with e-mail applications, with the Windows open file dialog, and in other, more specialized applications. We also explored dynamic search as a method of browsing, and ways that it could be integrated with filing systems. These concepts could radically change electronic storage systems, making them faster and more natural for users.
Related Work
This tool takes the idea of dynamic sliders for age and file size from, Browsing Unix Directories with Dynamic Queries: An Evaluation of Three Information Display Techniques.
Most of Apples iApps (iTunes for sure) include a dynamic search feature.
Scopeware Vision is built off of Microsoft Indexing Service as well. Vision allows users to search through e-mail as well as files. Since Indexing Service can only search through files, Vision duplicates a users inbox in the file system. A similar hack could be applied to make this dynamic query system operate on e-mail.
References
Browsing vs. Search: Can We Find a Synergy?E-mail as habitat: an exploration of embedded personal information management