Browse Books

The Web explosion offers a bonanza of novel problems. In particular, information retrieval in the Web context requires methods and ideas that have not been addressed in the classic information retrieval literature. This chapter will survey emerging ...

High-performance web crawlers are an important component of many web services. For example, search services use web crawlers to populate their indices, comparison shopping engines use them to collect product and pricing information from online vendors, ...

Internet traffic is approximately doubling each year. This growth rate applies not only to the entire Internet, but to a large range of individual institutions. For a few places we have records going back several years that exhibit this regular rate of ...

Many massive graphs (such as WWW graphs and Call graphs) share certain universal characteristics which can be described by the so-called the "power law". In this paper, we first briefly survey the history and previous work on power law graphs. Then we ...

We consider the task of evaluating properties of graphs that are too big to be even scanned. Thus, the input graph is given in form of an oracle which answers questions of the form is there an edge between vertices u and υ, or who is the i^th neighbor of ...

String Pattern Matching concerns itself with algorithmic and combinatorial issues related to matching and searching on linearly arranged sequences of symbols, arguably the simplest possible discrete structures. As unprecedented volumes of sequence data ...

In this chapter we present the main data structures and algorithms for searching large text collections. We emphasize inverted files, the most used index, but also review suffix arrays, which are useful in a number of specialized applications. We also ...

The exponential growth of computer applications in the last three decades of the 20th century has resulted in an explosive growth in the amounts of data moved between computers, collected, and stored by computer users. This, in turn, has created the ...

In many massive dataset applications the data must be stored in space and query efficient data structures on external storage devices. Often the data needs to be changed dynamically. In this chapter we discuss recent advances in the development of ...

Data sets in large applications are often too massive to fit completely inside the computer's internal memory. The resulting input/output communication (or I/O) between fast internal memory and slower external memory (such as disks) can be a major ...

Data Envelopment Analysis (DEA) is a clustering methodology for records in data sets corresponding to entities sharing a common list of attributes. Broadly defined, DEA partitions the records into two subsets; those 'efficient' and those 'inefficient.' ...

We describe the role of generalized support vector machines in separating massive and complex data using arbitrary nonlinear kernels. Feature selection that improves generalization is implemented via an effective procedure that utilizes a polyhedral ...

With the requirements of scientific and medical image database support in mind, we describe a range of useful technologies for storage, transmission and display. These new technologies are all based on discrete wavelet or related multiscale transforms. ...

We review the time and storage costs of search and clustering algorithms. We exemplify these, based on case-studies in astronomy, information retrieval, visual user interfaces, chemical databases, and other areas. Theoretical results developed as far ...

Data cubes combine an easy-to-understand conceptual model with an implementation that enables the fast summarization of large data sets. This makes them a powerful tool for supporting the interactive analysis of massive data collections like data ...

A "large dataset" is here defined as one that cannot be analyzed using some particular desired combination of hardware and software because of computer memory constraints. DuMouchel et al. (1999) defined "data squashing" as the construction of a ...

Data mining algorithms have been the focus of much recent research. The initial spurt of research on data mining algorithms typically considered static datasets. In practice, the input data to a data mining process resides in a large data warehouse ...

All data contain errors, and large spatial data sets are especially prone because they contain data from multiple sources, and use different assumptions about structure and semantics. The general issue is one of data quality assurance, defined in terms ...

A data warehouse is a repository for information that is collected, cleaned, and made available for analysis. A well run data warehouse makes many analyses easy to run, because many complex details have been taken care of already. In this chapter, we ...

Materialized views and their potential have been recently rediscovered for the content of OLAP and data warehousing. A flurry of papers has been generated on how views can be used to accelerate ad-hoc computations over massive datasets. In this chapter ...

The distinguishing feature of semistructured data is that the schema is embedded with the data. The main challenge is to cope with the additional flexibility without sacrificing efficiency. We introduce semistructured data by presenting a syntax and ...

The overview given here concentrates on the computational capabilities of the systems discussed. To do full justice to all assets of present days high-performance computers one should list their I/O performance and their connectivity possibilities as ...

We discuss three principles learned from experience with the National Scalable Cluster Project. Storing, managing and mining massive data requires systems that exploit parallelism. This can be achieved with shared-nothing clusters and careful attention ...

Data explosion is an increasingly prevalent problem in every field of science. Traditional out-of-core models that assume a single disk have been found inadequate to handle voluminous data. As a result, models that employ multiple disks have been ...

There is a growing need for very large databases which are not practical to implement with conventional relational database technology. These databases are characterized by huge size and frequent large updates; they do not require traditional database ...

Finding telecommunications fraud in masses of call records is more difficult than finding a needle in a haystack. In the haystack problem, there is only one needle that does not look like hay, the pieces of hay all look similar, and neither the needle ...

Astronomy has a long history of acquiring, systematizing, and interpreting large quantities of data. Starting from the earliest sky atlases through the first major photographic sky surveys of the 20th century, this tradition is continuing today, and at ...

This chapter describes the design and implementation of information systems to support decision-making in environmental management and protection. We employ a three-way object model: An environmental object (such as a lake) is described by one or more ...

Current and next decade global Earth observing, other remote sensing and related climate analysis data collected by space and operational U.S. agencies such as NASA and NOAA, the European ESA, the Japanese NASDA and other international agency missions ...

Biomolecular data mining is the activity of finding significant information in protein, DNA and RNA molecules. The significant information may refer to motifs, clusters, genes, protein signatures and classification rules. This chapter presents an ...

Air pollution, especially the reduction of the air pollution to some acceptable levels, is a highly relevant environmental problem, which is becoming more and more important. This problem can successfully be studied only when high-resolution ...