Comments on Stanford InfoBlog: Why Uncertainty in Data is Great (Posted by Anish ...

Interesting post. I have been wondering about this...

2009-11-21T19:23:36.888-08:00

Interesting post. I have been wondering about this issue,so thanks for posting. I’ll likely be coming back to your blog. Keep up great writing. Find your great Travel News and sing the songs at Free Song Lyric or you can watch the drama at Korea Drama Online one of great korea drama is A Love to Kill if you go to travel to Indonesia learn Learn Indonesia Language first! And find your home cari rumah or make a blog Belajar membuat Blog find your home again rumah dijual and again at jual rumah or something like download youtube or you can find a nice widget blog then if you want buy a new laptop see the Laptop Price List or you can buy a New Blackberry and then take care your Health & Jewerly, that's all, thank you so much.

Thanks ever so much, very useful article. Great in...

2009-10-12T20:35:11.359-07:00

I found your blog on google and read a few of your...

2009-10-04T00:34:01.600-07:00

I found your blog on google and read a few of your other posts. I just added you to my Google News Reader. Keep up the good work. Look forward to reading more from you in the future.

iklan baris gratis | jaringan iklan gratis baris | iklan baris gratis | pasang iklan baris gratis | submit iklan baris gratis | media pasang iklan gratis | promosi gratis iklan baris gratis | iklan baris gratis | pasang iklan baris gratis

Nice post. I just stumbled upon your blog and want...

2009-09-26T22:20:15.326-07:00

Nice post. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I'll be subscribing to your feed and I hope you post again soon.

if you do not mind, please visit my article related to pandeglang district in Banten, Indonesia at Kenali dan Kunjungi Objek Wisata di Pandeglang and also related to a leadership at Mengembalikan Jati Diri Bangsa

Oes Tsetnoc | Oes Tsetnoc

thanks for this usefull informations.. now i find ...

2009-09-25T07:42:38.363-07:00

thanks for this usefull informations..
now i find what i want to know..
thanks..
Kenali dan Kunjungi Objek Wisata di Pandeglang | morat marit | cah bagoes | oes tsetnoc

I agree if this article is very nice...absolutely ...

2009-09-07T21:00:14.346-07:00

I agree if this article is very nice...absolutely agree with you..thx for sharing.

Mengembalikan Jati Diri Bangsa | Kenali dan Kunjungi Objek Wisata di Pandeglang

A very nice illustration on the significance of ma...

2008-12-09T23:58:00.000-08:00

A very nice illustration on the significance of managing uncertain data; I have worked in the past in this area on evaluating threshold based preference queries on uncertain data; but we modeled uncertain data as ranges; For example instead of saying, I have a confidence of 80% on value X, I would say, the value is somewhere in the range [a,b] where this range could again follow an arbitrary distribution'; In the case of sensor networks, it mostly would follow a Gaussian distribution. There has been some other related work in the same lines by some DB groups in Purdue, Hong Kong University, Toronto too.

You are absolutely that propagating probabilities ...

2008-08-01T13:44:00.000-07:00

You are absolutely that propagating probabilities have been considered in AI (as well as in other fields), and we are aware of this past work. However, note that the uncertain data management we (and other DB groups around the globe) are doing is different in several respects. First, we consider more general kinds of uncertainty, which also includes non-probabilistic but uncertain data, and probably at a larger scale. Second, the kinds of queries that need to be answered in a relational setting (all of SQL) goes beyond Bayesian inference. And for these queries, we need to consider new kinds of indexes and statistics, etc.

That said, I would like to point out that within the DB community itself there has been past work that uses AI-ish techniques for managing relation uncertain data (See this ICDE 2007 and VLDB 2008 paper).

Finally, note that uncertainty is only one component of the Trio project I referred to. Data lineage constitutes the other key component in Trio. We've been looking at how lineage can play a crucial role in improving the efficiency and usability in uncertain data. (Our ICDE 2008 paper shows how lineage can help in confidence computation, and this paper shows that lineage can greatly simplify data modifications and versioning.)

As for your note on my Toucan example on how to "reconcile" independent sources of uncertain data, thanks for pointing me to this past work! Along with other members of the Trio project, I've been doing some work on building a theoretical framework that allows us to combine such uncertain information. Our work is based on and extends the theory of data integration, which fundamentally relies on the notion of containment. We believe the approach we are taking is more principled. Unfortunately I can't point you to our results yet, as we haven't published them.

Thanks a lot for your comments and for pointing me to your past and current work!

Propagating uncertainty is a cornerstone of Bayesi...

2008-07-29T15:30:00.000-07:00

Propagating uncertainty is a cornerstone of Bayesian statistics, which provides a general framework for uncertainty integration. Some of your colleagues in CS at Stanford are using it for a general model of natural language processing that propagates uncertainty through a linguistic processing pipeline (see this paper).

Back in the 1980s, I used to do logic programming and knowledge representation for computational linguistics, with a theoretical focus on uncertainty propagation (though it was disjunctive or inheritance-based uncertainty).

Fast forward to the 2000s, and I'm currently working on an NIH grant, the foundation of which is high recall techniques for linking textual mentions of genes, mutations, diseases and other biological entities to databases (see this tech report, blog entry or tutorial). You just can't get high recall with state of the art data cleaning in 2008, at least in the domains we care about.

A related issue I'm working on now is a hierarchical Bayesian model of determining true annotations from multiple annotators (like your Toucan example). I discuss the general problems with the current way of measuring agreement for producing gold standard data in my last two blog entries, Good Kappa's Not Enough and Good Kappa's Not Necessary, Either.

Finally, it's not just unclean data that needs to be reasoned with, but also missing data, which presents a different set of problems. For that, you might consider multiple imputation.