Adequately describes how I’ve felt the last couple weeks.

Millions of miles away, the Curiosity rover is going through an exhaustive battery of tests to make sure everything’s kosher before embarking on what will almost surely be the most thorough and informative Mars exploration to date.

Meanwhile, we’re stuck on this planet with morons like this guy (irony of ironies: he’s on the Congressional Science Committee. NO JOKE).

In case I’ve been too subtle: yes, it’s been a ball-busting couple of weeks in the ‘burgh. Work has been extremely nose-to-the-grindstone, though I’m optimistic about how the impending fall semester will pan out. Nevertheless, here is a glimpse into the work that’s been keeping me up late:

Yep!

Ok but really: I’ve been working on implementing a shiny new semi-supervised clustering algorithm, the basic idea being that we know what some of the data is, but the vast majority of it is of an unknown nature. In the case of the images I’m using to test it: can I separate the foreground from the background? The catch here is that I’m giving the algorithm the entire image, but only a handful of pixels are labeled as actual “foreground” or “background” pixels; it has to figure out the rest on its own.

What’s super cool about machine learning with images is that you get stuff like this:

Those familiar with this sort of thing will likely recognize the above images as eigenvectors of the original image. For any sort of spectral analysis where graphs are involved, finding eigenvectors is pretty much par for the course. It’s how you use the eigenvectors that makes each algorithm distinct. I’ve been working on implementing this one, ideally smoothing its implementation to the point where I can submit it to these guys.

And then put it into Mahout. But that’s still a ways away (and I have other work to do there that’s more pressing than adding new algorithms).

Ultimately, I get this:

It’s not ideal, particularly since the few labeled pixels I handed off to the algorithm included the bowl as “background”, and yet the algorithm decided it knew better than I did and included it as foreground. So there are still some things to iron out.

But this is kinda neat!

We’re hoping to get a paper out in the next couple weeks on this. Sadly, it won’t involve cats; rather, our goal is to deploy it on small molecules. While I’m not structural biology’s biggest fan (read: I leave the seminar when I learn it’s on structural biology), this particular application is really interesting to me.

And yes, one of my to-dos is to optimize the algorithm a bit more, so it can handle images that are bigger than little thumbnails. As in most spectral algorithms, one needs to create a graph matrix that is #pixels-by-#pixels. For 50×50 images (like the little cat above), this is only 2500 total pixels, resulting in a graph matrix that’s 2500×2500. That’s only 6.2 million numbers, which will fit in less than 1GB of memory. But for even slightly larger images–say, 640×480–now the number of pixels is 307k, resulting in a graph matrix that holds more than 9.4 billion numbers. Most home computers right now only have 4 or 8GB of memory, and would instantly freeze when attempting to create the graph matrix.

HOORAY MATHS!

Other exciting news? September is almost here, which meaaaaaans:

• Sept 1: Zombie 5k
• Sept 15: USAF half marathon
• Sept 21-22: D.C Ragnar
• Sept 30: Great Race 10k

I am trying to get back on a regular posting regimen. I’m thinking once a week on the same day each week. How do Tuesdays sound?

When the stress levels get too high, I just look at this picture. Enjoy!

We approach the problem of semi-supervised shape classification by exploiting the geometric structure of shape data. We apply manifold regularization to learn a function from shapes to class labels. Central to manifold regularization algorithms is the use of a weighted graph to represent pairwise relationships between training points and capture their geometric structure. Under a regularized least squares formulation, each algorithm only involves solving a linear system of equations. We analyze the classification performance for different features regularization parameters, percentage of labeled data, and noise in the labels. We show that encouraging the smoothness of the classification function on the manifold improves classification performance beyond simply encouraging smoothness in the ambient space. We compare raw pixel features to Signed Distance Function (SDF) features and find that SDF features yield consistently higher accuracy. Finally, we compare manifold regularization to the much simpler k-Nearest-Neighbors and show that manifold regularization is consistently better.

Keywords: shape classification, manifold regularization, semi-supervised learning.

homepage

Semi-Supervised Learning in Vision, CVPR 2010 Tutorial

We are organizing a tutorial at CVPR 2010 on "Semi-Supervised Learning in Vision". The following is a brief introduction and a set of topics we will cover in the tutorial. Please give us your feedback regarding this tutorial, for example, if you would like to see a method or topic which is not covered by the following list, or if you would like to see a topic discussed in more details, please let us know.

We also plan to publish a set of open/closed source software available for semi-supervised learning. So if you would like to have your package included here, please send us a short description of your method and a link to where it can be obtained.

Introduction

Current supervised approaches obtain high recognition rates if enough labeled training data is available. However, for most practical problems there is simply not enough labeled data available, whereas hand-labeling is tedious and expensive, in some cases not even feasible. This is especially true for applications in computer vision like object recognition and categorization from images and videos, where the human effort is needed to determine the true contents of the media. Semi-supervised methods offer an interesting solution to this problem by learning from both labeled and unlabeled data. These methods try to give an answer to the question: “How to improve classification accuracy using unlabeled data together with the labeled data?”.

This course will cover an introduction to semi-supervised learning, its applications in computer vision, and the open problems and challenges facing the future research in this field. Nowadays, the Internet offers a huge amount of data in form of unlabeled data (or labeled with high degree of uncertainty), and learning from Internet is becoming more and more widespread in computer vision. Therefore, we will have a special focus on issues dealing with large-scale and on-line semi-supervised learning tasks.

The first half of the course will address the basics of semi-supervised learning and its relations to other machine learning domains. It will cover the major works in this field from a unified point of view, and will discuss the advantages and disadvantages of these methods from theoretical and application perspectives. In the second part, we will focus on the applications of the semi-supervised learning in computer vision, and the open challenges and gaps in existing methods.

Slides

1. Introductions and motivations: Horst Bischof [slides] (the slides on slideshare.net)
2. Theory of Semi-Supervised Learning: Amir Saffari [slides] (the slides on slideshare.net)
3. Algorithms and Applications: Christian Leistner [slides] (the slides on slideshare.net)

Detailed Outline

   1. Motivations.
   2. A unified perspecive on learning from unlabeled data.
   3. Statistical models for semi-supervised learning.
   4. Semi-supervised learning with margins.
   5. Methods based on manifold learning.
   6. Co-training and learning from multiple-views.
   7. Semi-supervised learning for large-scale problems.
   8. The relations to other fields: unsupervised learning, one-shot learning, transfer learning, and multiple instance learning.
   9. Applications: object recognition and categorization.
  10. Applications: object segmentation.
  11. Applications: object tracking.
  12. Applications: activity recognition.
  13. Open problems and challenges.

Target Audience

The course will be interesting for those who investigate or apply learning methods in computer vision. The course is directed towards the researchers, practitioners and PhD students working on related topics. The tutorial is self-contained in the sense that it requires a minimum knowledge of basic mathematical concepts, such as statistics and linear algebra.

Organizers

• Amir Saffari, Graz University of Technology
• Christian Leistner, Graz University of Technology
• Horst Bischof, Graz University of Technology

[arXiv:stat.ME:0910.2585]
Variable Selection and Updating In Model-Based Discriminant Analysis for High Dimensional Data with Food Authenticity Applications
by Murphy, Dean, and Raftery

Classifying or clustering (or semi supervised learning) spectra is a very challenging problem from collecting statistical-analysis-ready data to reducing the dimensionality without sacrificing complex information in each spectrum. Not only how to estimate spiky (not differentiable) curves via statistically well defined procedures of estimating equations but also how to transform data that match the regularity conditions in statistics is challenging.

Another reason that astrophysics spectroscopic data classification and clustering is more difficult is that observed lines, and their intensities and FWHMs on top of continuum are related to atomic database and latent variables/hyper parameters (distance, rotation, absorption, column density, temperature, metalicity, types, system properties, etc). Frequently it becomes very challenging mixture problem to separate lines and to separate lines from continuum (boundary and identifiability issues). These complexity only appears in astronomy spectroscopic data because we only get indirect or uncontrolled data ruled by physics, as opposed to the the meat species spectra in the paper. These spectroscopic data outside astronomy are rather smooth, observed in controlled wavelength range, and no worries for correcting recession/radial velocity/red shift/extinction/lensing/etc.

Although the most relevant part to astronomers, i.e. spectroscopic data processing is not discussed in this paper, the most important part, statistical learning application to complex curves, spectral data, is well described. Some astronomers with appropriate data would like to try the variable selection strategy and to check out the classification methods in statistics. If it works out, it might save space for storing spectral data and time to collect high resolution spectra. Please, keep in mind that it is not necessary to use the same variable selection strategy. Astronomers can create better working versions for classification and clustering purpose, like Hardness Ratios, often used to reduce the dimensionality of spectral data since low total count spectra are not informative in the full energy (wavelength) range. Curse of dimensionality!.

I will argue that the view-redundancy assumption is unnecessary, and along the way show how surrogate learning can be plugged into co-training  (which is not all that surprising considering that both are multi-view semi-sup algorithms that rely on class-conditional view-independence).

I’ll first explain co-training with an example.

Co-training – The setup

Consider a classification problem on the feature space . I.e., a feature vector can be split into two as .

We make the rather restrictive assumption that and are class-conditionally independent for both classes. I.e., for .

(Note that unlike surrogate learning with mean-independence, both  and are allowed to be multi-dimensional.)

Co-training makes an additional assumption that either view is sufficient for classification. This view-redundancy assumption basically states that the probability mass in the region of the feature space, where the Bayes optimal classifiers on the two views disagree with each other, is zero.

(The original co-training paper actually relaxes this assumption in the epilogue, but it is unnecessary to begin with, and the assumption has proliferated in later manifestations of co-training.)

We are given some labeled data (or a weak classifier on one of the views) and an large supply of unlabeled data. We are now ready to proceed with co-training to construct a Bayes optimal classifier.

Co-training – The algorithm

The algorithm is very simple. We use our weak classifier, say , (which we were given, or which we constructed using the measly labeled data) on the one view () to classify all the unlabeled data.  We select the examples classified with high confidence, and use these as labeled examples (using the labels assigned by the weak classifier) to train a classifier on the other view ().

We now classify the unlabeled data with to similarly generate labeled data to retrain . This back-and-forth procedure is repeated until exhaustion.

Under the above assumptions (and with “sufficient” unlabeled data) and converge to the Bayes optimal classifiers on the respective feature views. Since either view is enough for classification, we just pick one of the classifiers and release it into the wild.

Co-training – Why does it work?

I’ll try to present an intuitive explanation of co-training using the example depicted in the following figure. Please focus on it intently.

The feature vector in the example is 2-dimensional and both views and  are  1-dimensional. The class-conditional distributions are uncorrelated and jointly Gaussian (which means independent) and depicted by their equiprobability contours in the figure. The marginal class-conditional distributions are show along the two axes. Class is shown in red and class is shown in blue. The picture also shows some unlabeled examples.

Assume we have a weak classifier on the first view. If we extend the classification boundary for this classifier to the entire space ,  the boundary necessarily comprises of lines parallel to the axis.  Let’s say there is only one such line and all the examples below that line are assigned class and all the examples above are assigned class .

We now ignore all the examples close to the classification boundary of  (i.e., all the examples in the grey band) and project the rest of the points onto the axis.

How will these projected points be distributed along ?

Since the examples that were ignored (in the grey band) were selected based on their values, owing to class-conditional independence, the marginal distribution along for either class will be exactly the same as if none of the samples were ignored. This is the key reason for the conditional-independence assumption.

The procedure has two subtle, but largely innocuous, consequences.

First, since we don’t know how many class and class examples are in the grey band the relative ratio of the examples of the two classes in the not-ignored set may not the same as in the original full unlabeled sample set. If the class priors are known, this can easily be corrected for when we learn . If the class priors are unknown other assumptions on are necessary.

Second, when we project the unlabeled examples on to we assign them the labels given to them by which can be erroneous. In the figure above, there will be examples in the region indicated by A that are actually class but have been assigned class , and examples in region B that were from class but were called class .

Again because of the class-conditional independence assumption these erroneously labeled examples will be distributed according to the marginal class-conditional distributions. I.e., in the figure above we imagine, along the axis, a very low amplitude blue distribution with the same shape and location as the red distribution, and a very low amplitude red distribution with the same shape under the blue distribution. (Note . This is the noise in the original co-training paper.)

This amounts to having a labeled training set with label errors but with errors being generated independently of the location in the space. That is the number of errors in a region in the space is proportional to the number of examples in that region. These proportionally distributed errors are then washed out by the correctly labeled examples when we learn .

To recap, co-training works because of the following fact. Starting from a weak classifier on , we can generate very accurate and unbiased training data to train a classifier on .

No need for view-redundancy

Notice that, in the above example, we made no appeal to any kind of view-redundancy (other than whatever we may get gratis from the independence assumption).

The vigilant reader may however level the following two objections against the above argument-by-example.

1. We build and separately. So when the training is done, without view redundancy, we have not shown a way to pick from the two to apply to new test data.

2. At every iteration we need to select unlabeled samples that were classified with high-confidence by to feed to the trainer for . Without view-redundancy may be none of the samples will be classified with high confidence.

The first objection is easy to respond to. We pick neither nor for new test data. Instead we combine them to obtain a classifier . This is well justified because, under class-conditional independence, .

We react to the second objection by dropping the requirement of classifying with high-confidence altogether.

Dropping the high-confidence requirement by surrogate learning

Instead of training with examples that are classified with high confidence by , we train with all the examples (using the scores assigned to them by ).

At some iteration of co-training, define the random variable . Since and are class-conditionally independent, and are also class-conditionally independent. In particular  is class-conditionally mean-independent of . Furthermore if is even a weakly useful classifier, barring pathologies, it will satisfy .

We can therefore apply surrogate learning under mean-independence to learn the classifier on . (This is essentially the same idea as Co-EM, which was introduced without much theoretical justification.)

Discussion

Hopefully the above argument has convinced the reader that the class-conditional view independence assumption obviates the view-redundancy requirement.

A natural question to ask is whether the reverse is true. That is, if we are given view-redundancy, can we completely eliminate the requirement of class-conditional independence? We can immediately see that the answer is no.

For example, we can duplicate all the features for any classification problem so that view-redundancy holds trivially between the two replicates. Moreover, the second replicate will be statistically fully dependent on the first.

Now if we are given a weak classifier on the first view (or replicate) and try to use its predictions on an unlabeled data set to obtain training data for the second, it would be equivalent to feeding back the predictions of a classifier to retrain itself (because the two views are duplicates of one another).

This type of procedure (which is an idea decades old) has been called, among other things, self-learning, self-correction, self-training and decision-directed adaptation. The problem with these approaches is that the training set so generated is biased and other assumptions are necessary for the feedback procedure to improve over the original classifier.

Of course this does not mean that the complete statistical independence assumption cannot be relaxed. The above argument only shows that at least some amount of independence is necessary.

Murders she induced

For readers who appreciate this sort of a thing, here’s the grandmother of all murder mysteries. In this one Miss Marple solves a whole bunch of unrelated murders all at once.

Miss Marple having finally grown wise to the unreliability of feminine intuition in solving murder mysteries spent some time learning statistics. She then convinced one of her flat-footed friends at the Scotland Yard to give her the files on all the unsolved murders, that were just sitting around gathering dust and waiting for someone with her imagination and statistical wiles.

She came home with the massive pile of papers and sat down to study them.

The file on each murder carefully listed all the suspects, with their possible motives, their accessibility to the murder weapon, psychological characteristics, previous convictions and many other details. There was a large variety in the number of suspects for different murders.

Furthermore, for every case the investigating officer made a note that the murderer was very likely to be part of the suspect pool. Only, it was not possible to narrow down the pool to one, and none of the suspects confessed.

Thinking statistically, Miss Marple decided to encode the characteristics of each suspect by a feature vector. Drawing on her vast experience in these matters she assigned numeric values to features like motives, psychology, relationship to the victim and access to the weapon.

All that was left was to build a statistical model that predicted the probability of a suspect being the murderer from his feature vector.  She would then rank all the suspects of a case by this probability, and then she can have Scotland Yard try to get the most likely suspect to confess.

At this point Miss Marple paused for a moment to rue her oversight in not asking for the files on the solved murders as well. She could have used the suspects (and the known murderer) from the solved murders as labeled examples to train her statistical model. However, she soon realized …

I know you are tense with anticipation of the inevitable twist in the tale. Here it is.

… she soon realized that she could build her model by training a regressor to predict, from the feature vector of a suspect, the size of the suspect pool he belongs to.

The clues

Let’s re-examine the pertinent facts — 1. every suspect pool contains the murderer, 2.  the suspect pools are of varying sizes.

The one, perhaps controversial, assumption we’ll need to make is that the feature vectors for both the murderers and the non-murderers are mean-independent of the size of the suspect pool they come from. (However, if it is good enough for Miss Marple, it is good enough for me.)

In any case, the violation of this assumption would imply that, when whether or not a suspect is a murderer is known, knowing his feature vector allows us to better predict the size of the suspect pool he belongs to. Because this is far-fetched, we can believe that the assumption is true.

In fact it may seem that knowing whether or not a suspect is murderer itself adds nothing to the predictability of the size of the pool from which the suspect is drawn. We will however show that the means for the murderers and non-murderers is different.

Let us now see why Miss Marple’s method works.

Marple math

Let the suspect pools be denoted with sizes . Let each suspect be described by the feature vector denoted . We now define the class label if a particular suspect is a murderer and otherwise.

We append the feature to the feature vector of a suspect from pool . That is, the surrogate feature we use is the negative of the pool size. (The negative is for a technical reason to match the previous post. In practice I would recommend , but this makes the argument below a bit more involved.)

The first thing to note is that the assumption we made above translates to for both and .

All we need to show is that and we are home free, because the argument in the previous post implies that can be used to rank suspects in a pool just as well as .

So all we need to do is to build a regressor to predict from the feature vector and apply this regressor to the of all the suspects in each pool and rank them by its output.

So why is ?

First it is clear that (where is the random variable describing the size of the suspect pool) because for each pool there is one murderer and we assign the negative of that pool size as for that murderer.

Now there are a total of non-murderers. For the non-murderers from the pool the assigned is . Therefore the estimated expected value is

If we divide through by and let the number of pools go to infinity, the estimate converges to the true mean, which is

It is easy to show that this quantity is always lower than if the variance of is non-zero (and if ), which we were given by the fact that the pool sizes are variable across murders.

And there you have it.

A less criminal example

Let me try to demystify the approach even more.

Assume that we have a database of images of faces of people along with their last names. Now the problem is the following — for a given face-name input pair to find the matching face and name in the database. Of course if all the names are unique this is trivial, but there are some common names like Smith and Lee.

Let us say that our procedure for this matching is to first obtain from the database the face images of the people with the same last name as the input, and then using our state-of-the-art face image comparison features to decide which of those is the right one. Do we need a training set of human-annotated input-database record pairs to train such a ranker? The above discussion suggests not.

Let us say we create a “fake” training set from a lot of unlabeled input image-name pairs by getting the set of records from the database with the matching name and assigning each feature vector a label which is the reciprocal of the size of the set. We then learn to predict this label from the image features.

The reason we would expect this predictor to behave similarly to the predictor of image match is that we can view our fake labeling as a probabilistic labeling of the feature vectors. For a given set our label is the a priori probability of a face image being the right match to the input image.

The independence assumption just makes sure that our probabilistic labeling is not biased.

I am curious about other possible applications.

Similar ideas can be found in Ando and Zhang ’07, Quadrianto et. al. ’08, Blitzer et. al. ’06, and others.

Derivation from mean-independence

I’ll now derive a similar surrogate learning algorithm from mean independence rather than full statistical independence. Recall that the random variable is mean-independent of  the r.v. if . Albeit weaker than full independence, mean-independence is still a pretty strong assumption. In particular it is stronger than the lack of correlation.

We assume that the feature space contains the single feature and the rest of the features . We are still in a two-class situation, i.e., . We further assume

1. is at least somewhat useful for classification, or in other words, .

2. is class-conditionally mean-independent of , i.e., for .

Now let us consider the quantity . We have

Notice that  is a convex sum of and .

Now using the fact that  we can show after some algebra that

We have succeeded in decoupling and on the right hand side, which results in a simple semi-supervised classification method. We just need the class-conditional means of and a regressor (which can be learned on unlabeled data) to compute .  Again acting as a surrogate for is predicted from .

As opposed to the formulation in the paper this formulation easily accommodates continuous valued .

Discussion

1. The first thing to note is that we are only able to write an expression for but not . That is, we are able to weaken the independence to mean-independence at the expense of “wasting” feature .

Of course if we have full statistical independence we can use , by using Equation (1) and the fact that, under independence, we have

.

2. If (without loss of generality) we assume that , because lies somewhere between and , Equation (1) says that is a monotonically increasing function of .

This means that itself can be used as the classifier, and labeled examples are needed only to determine a threshold for trading off precision vs. recall. The classifier (or perhaps we should call it a ranker) therefore is built entirely from unlabeled samples.

I’ll post a neat little application of this method soon.

]]> http://hyunsook.wordpress.com/2008/09/18/classification-and-clustering/ Thu, 18 Sep 2008 23:48:47 +0000 HLee http://hyunsook.wordpress.com/2008/09/18/classification-and-clustering/ Another deduced conclusion from reading preprints listed in arxiv/astro-ph is that astronomers tend to confuse classification and clustering and to mix up methodologies. They tend to think any algorithms from classification or clustering analysis serve their purpose since both analysis algorithms, no matter what, look like a black box. I mean a black box as in neural network, which is one of classification algorithms.

Simply put, classification is regression problem and clustering is mixture problem with unknown components. Defining a classifier, a regression model, is the objective of classification and determining the number of clusters is the objective of clustering. In classification, predefined classes exist such as galaxy types and star types and one wishes to know what prediction variables and their functional allow to separate Quasars from stars without individual spectroscopic observations by only relying on handful variables from photometric data. In clustering analysis, there is no predefined class but some plots visualize multiple populations and one wishes to determine the number of clusters mathematically not to be subjective in concluding remarks saying that the plot shows two clusters after some subjective data cleaning. A good example is that as photons from Gamma ray bursts accumulate, extracting features like F_{90} and F_{50} enables scatter plots of many GRBs, which eventually led people believe there are multiple populations in GRBs. Clustering algorithms back the hypothesis in a more objective manner opposed to the subjective manner of scatter plots with non statistical outlier elimination.

However, there are challenges to make a clear cut between classification and clustering both in statistics and astronomy. In statistics, missing data is the phrase people use to describe this challenge. Fortunately, there is a field called semi-supervised learning to tackle it. (Supervised learning is equivalent to classification and unsupervised learning is to clustering.) Semi-supervised learning algorithms are applicable to data, a portion of which has known class types and the rest are missing — astronomical catalogs with unidentified objects are a good candidate for applying semi-supervised learning algorithms.

From the astronomy side, the fact that classes are not well defined or subjective is the main cause of this confusion in classification and clustering and also the origin of this challenge. For example, will astronomer A and B produce same results in classifying galaxies according to Hubble’s tuning fork? ((Check out the project, GALAXY ZOO)) We are not testing individual cognitive skills. Is there a consensus to make a cut between F9 stars and G0 stars? What make F9.5 star instead of G0? With the presence of error bars, how one is sure that the star is F9, not G0? I don’t see any decision theoretic explanation in survey papers when those stellar spectral classes are presented. Classification is generally for data with categorical responses but astronomer tend to make something used to be categorical to continuous and still remain to apply the same old classification algorithms designed for categorical responses.

From a clustering analysis perspective, this challenge is caused by outliers, or peculiar objects that do not belong to the majority. The size of this peculiar objects can make up a new class that is unprecedented before. Or the number is so small that a strong belief prevails to discard these data points, regarded as observational mistakes. How much we can trim the data with unavoidable and uncontrollable contamination (remember, we cannot control astronomical data as opposed to earthly kinds)? What is the primary cause defining the number of clusters? physics, statistics, astronomers’ experience in processing and cleaning data, …

Once the ambiguity in classification, clustering, and the complexity of data sets is resolved, another challenge is still waiting. Which black box? For the most of classification algorithms, Pattern Recognition and Machine Learning by C. Bishop would offer a broad spectrum of black boxes. Yet, the book does not include various clustering algorithms that statisticians have developed in addition to outlier detection. To become more rigorous in selecting a black box for clustering analysis and outlier detection, one is recommended to check,

• Clustering Analysis by Everitt, Landau, and Leese
• Data Clustering: Theory, Algorithms, and Applications by Gan, Ma, and Wu
• collection of articles and presentation files on nonparametric multivariate analysis by Robert Serfling (Yes, the author of the classical book, Approximation Theorems of Mathematical Statistics), particularly about data depth and outlier detection and
• robust statistics by Peter Huber

For me, astronomers tend to be in a haste owing to the pressure of publishing results immediately after data release and to overlook suitable methodologies for their survey data. It seems that there is no time for consulting machine learning specialists to verify the approaches they adopted. My personal prayer is that this haste should not be settled as a trend in astronomical survey and large data analysis.