Welcome Guest [Log In] [Register]


Reply
  • Pages:
  • 1
Not Even Scientists Can Easily Explain P-values
Topic Started: 18 Jan 2017, 02:48 PM (1,231 Views)
Rufus
Member Avatar


Not Even Scientists Can Easily Explain P-values
Link @ Five Thirty Eight

P-values have taken quite a beating lately. These widely used and commonly misapplied statistics have been blamed for giving a veneer of legitimacy to dodgy study results, encouraging bad research practices and promoting false-positive study results.

But after writing about p-values again and again, and recently issuing a correction on a nearly year-old story over some erroneous information regarding a study’s p-value (which I’d taken from the scientists themselves and their report), I’ve come to think that the most fundamental problem with p-values is that no one can really say what they are.

Last week, I attended the inaugural METRICS conference at Stanford, which brought together some of the world’s leading experts on meta-science, or the study of studies. I figured that if anyone could explain p-values in plain English, these folks could. I was wrong.

To be clear, everyone I spoke with at METRICS could tell me the technical definition of a p-value — the probability of getting results at least as extreme as the ones you observed, given that the null hypothesis is correct — but almost no one could translate that into something easy to understand.

It’s not their fault, said Steven Goodman, co-director of METRICS. Even after spending his “entire career” thinking about p-values, he said he could tell me the definition, “but I cannot tell you what it means, and almost nobody can.” Scientists regularly get it wrong, and so do most textbooks, he said. When Goodman speaks to large audiences of scientists, he often presents correct and incorrect definitions of the p-value, and they “very confidently” raise their hand for the wrong answer. “Almost all of them think it gives some direct information about how likely they are to be wrong, and that’s definitely not what a p-value does,” Goodman said.

We want to know if results are right, but a p-value doesn’t measure that. It can’t tell you the magnitude of an effect, the strength of the evidence or the probability that the finding was the result of chance.

So what information can you glean from a p-value? The most straightforward explanation I found came from Stuart Buck, vice president of research integrity at the Laura and John Arnold Foundation. Imagine, he said, that you have a coin that you suspect is weighted toward heads. (Your null hypothesis is then that the coin is fair.) You flip it 100 times and get more heads than tails. The p-value won’t tell you whether the coin is fair, but it will tell you the probability that you’d get at least as many heads as you did if the coin was fair. That’s it — nothing more. And that’s about as simple as I can make it, which means I’ve probably oversimplified it and will soon receive exasperated messages from statisticians telling me so.

What I learned by asking all these very smart people to explain p-values is that I was on a fool’s errand. Try to distill the p-value down to an intuitive concept and it loses all its nuances and complexity, said science journalist Regina Nuzzo, a statistics professor at Gallaudet University. “Then people get it wrong, and this is why statisticians are upset and scientists are confused.” You can get it right, or you can make it intuitive, but it’s all but impossible to do both.
Take risks - if you win you will become wealthy, if you lose you will become wise
Profile "REPLY WITH QUOTE" Go to top
 
Rufus
Member Avatar


Come on Terry, this is your chosen subject.
Surely you have an opinion.....
Take risks - if you win you will become wealthy, if you lose you will become wise
Profile "REPLY WITH QUOTE" Go to top
 
Terry
Member Avatar


Rufus
18 Jan 2017, 03:54 PM
Come on Terry, this is your chosen subject.
Surely you have an opinion.....
Good opinion. Unfortunately and fortunately, we need statistics, not to explain physical and non-physical, but to understand relationships between and within sets of data.
Profile "REPLY WITH QUOTE" Go to top
 
Mallard
Default APF Avatar
Thread killer

The problem is not with p values but on the randomness of the sample, or more commonly the lack of randomness. Surveys contact a weird bunch of people that don't say what they think
Collecting desperation.
Ex-Bp Golly April 2 2015. "I see with a slight overshoot -70% [fall in Sydney house prices] as being well within possibility"
Profile "REPLY WITH QUOTE" Go to top
 
Khaderbhai
Member Avatar
Wealthy Suburbanite

Terry
18 Jan 2017, 06:02 PM
Good opinion. Unfortunately and fortunately, we need statistics, not to explain physical and non-physical, but to understand relationships between and within sets of data.
Roddy, if scientists can't easily explain P-values, what hope is there for the likes of you?
Banks can't repossess your home simply because the market value falls. Australia's Consumer Credit Code says consumers aren't liable for things ordinarily outside their control and can't be held to obligations that could only be met by selling their home. Click for details.

"The truth is that there are no good men, or bad men. It is the deeds that have goodness or badness in them. There are good deeds, and bad deeds. Men are just men."
Profile "REPLY WITH QUOTE" Go to top
 
Terry
Member Avatar


Mallard
18 Jan 2017, 06:17 PM
The problem is not with p values but on the randomness of the sample, or more commonly the lack of randomness. Surveys contact a weird bunch of people that don't say what they think
Nope. P values are not resigned to consumer surveys. They're use in literally million of purposes, including engineering and scientific research.
Khaderbhai
18 Jan 2017, 07:02 PM
Roddy, if scientists can't easily explain P-values, what hope is there for the likes of you?
P-values are used to understand relationships between and within sets of data mother cat.
Edited by Terry, 18 Jan 2017, 08:37 PM.
Profile "REPLY WITH QUOTE" Go to top
 
Khaderbhai
Member Avatar
Wealthy Suburbanite

Terry
18 Jan 2017, 08:35 PM
P-values are used to understand relationships between and within sets of data
So are lots of things Roddy. Is that the best you've got?
Banks can't repossess your home simply because the market value falls. Australia's Consumer Credit Code says consumers aren't liable for things ordinarily outside their control and can't be held to obligations that could only be met by selling their home. Click for details.

"The truth is that there are no good men, or bad men. It is the deeds that have goodness or badness in them. There are good deeds, and bad deeds. Men are just men."
Profile "REPLY WITH QUOTE" Go to top
 
Terry
Member Avatar


Khaderbhai
18 Jan 2017, 08:41 PM
So are lots of things Roddy. Is that the best you've got?
That's true MC. It probably wasn't so long ago you were unlikely even to contemplate what a p value is. Analytics can be spooky for the unitiated.
Edited by Terry, 18 Jan 2017, 11:11 PM.
Profile "REPLY WITH QUOTE" Go to top
 
skamy
Member Avatar


Rufus
18 Jan 2017, 02:48 PM
Not Even Scientists Can Easily Explain P-values
Link @ Five Thirty Eight

P-values have taken quite a beating lately. These widely used and commonly misapplied statistics have been blamed for giving a veneer of legitimacy to dodgy study results, encouraging bad research practices and promoting false-positive study results.

But after writing about p-values again and again, and recently issuing a correction on a nearly year-old story over some erroneous information regarding a study’s p-value (which I’d taken from the scientists themselves and their report), I’ve come to think that the most fundamental problem with p-values is that no one can really say what they are.

Last week, I attended the inaugural METRICS conference at Stanford, which brought together some of the world’s leading experts on meta-science, or the study of studies. I figured that if anyone could explain p-values in plain English, these folks could. I was wrong.

.....................
The problem I have found is that people use these statistics so badly. I once visited a Company who were doing recovery studies for a regulatory authority and they said that their results kept failing the test set up for them by a statistician.

All the regulatory body needed to know was that the recovery was above 96%. However the statistician set up a significance test to test if there was any difference between the amount originally added to sample and the amount measured at the end of a complex wet chemical analysis procedure.

The test was failing because it is obvious that there is going to be a difference between a 5 figure balance and a complex extraction and measurement process. Yet they repeated this test over and over hoping for an acceptable p value.

This was one classic example of silly use of statistics. All they needed to measure was the % recovery, but the analysts did not have the confidence to query the statistician.

Terry entertains us here regularly with a plethora of similar examples lol.
Definition of a doom and gloomer from 1993
The last camp is made up of the doom-and-gloomers. Their slogan is "it's the end of the world as we know it". Right now they are convinced that debt is the evil responsible for all our economic woes and must be eliminated at all cost. Many doom-and-gloomers believe that unprecedented debt levels mean that we are on the precipice of a worse crisis than the Great Depression. The doom-and-gloomers hang on the latest series of negative economic data.
Profile "REPLY WITH QUOTE" Go to top
 
Matthew
Default APF Avatar


Rufus
18 Jan 2017, 03:54 PM
Come on Terry, this is your chosen subject.
Surely you have an opinion.....
I will tell you who cannot easily explain anything, at all, ever - Terry.
My only hope for my three boys is that they turn out nothing at all like Chris.
Profile "REPLY WITH QUOTE" Go to top
 
1 user reading this topic (1 Guest and 0 Anonymous)
ZetaBoards - Free Forum Hosting
Join the millions that use us for their forum communities. Create your own forum today.
Go to Next Page
« Previous Topic · Australian Property Forum · Next Topic »
Reply
  • Pages:
  • 1



Australian Property Forum is an economics and finance forum dedicated to discussion of Australian and global real estate markets and macroeconomics, including house prices, housing affordability, and the likelihood of a property crash. Is there an Australian housing bubble? Will house prices crash, boom or stagnate? Is the Australian property market a pyramid scheme or Ponzi scheme? Can house prices really rise forever? These are the questions we address on Australian Property Forum, the premier real estate site for property bears, bulls, investors, and speculators. Members may also discuss matters related to finance, modern monetary theory (MMT), debt deflation, cryptocurrencies like Bitcoin Ethereum and Ripple, property investing, landlords, tenants, debt consolidation, reverse home equity loans, the housing shortage, negative gearing, capital gains tax, land tax and macro prudential regulation.

Forum Rules: The main forum may be used to discuss property, politics, economics and finance, precious metals, crypto currency, debt management, generational divides, climate change, sustainability, alternative energy, environmental topics, human rights or social justice issues, and other topics on a case by case basis. Topics unsuitable for the main forum may be discussed in the lounge. You agree you won't use this forum to post material that is illegal, private, defamatory, pornographic, excessively abusive or profane, threatening, or invasive of another forum member's privacy. Don't post NSFW content. Racist or ethnic slurs and homophobic comments aren't tolerated. Accusing forum members of serious crimes is not permitted. Accusations, attacks, abuse or threats, litigious or otherwise, directed against the forum or forum administrators aren't tolerated and will result in immediate suspension of your account for a number of days depending on the severity of the attack. No spamming or advertising in the main forum. Spamming includes repeating the same message over and over again within a short period of time. Don't post ALL CAPS thread titles. The Advertising and Promotion Subforum may be used to promote your Australian property related business or service. Active members of the forum who contribute regularly to main forum discussions may also include a link to their product or service in their signature block. Members are limited to one actively posting account each. A secondary account may be used solely for the purpose of maintaining a blog as long as that account no longer posts in threads. Any member who believes another member has violated these rules may report the offending post using the report button.

Australian Property Forum complies with ASIC Regulatory Guide 162 regarding Internet Discussion Sites. Australian Property Forum is not a provider of financial advice. Australian Property Forum does not in any way endorse the views and opinions of its members, nor does it vouch for for the accuracy or authenticity of their posts. It is not permitted for any Australian Property Forum member to post in the role of a licensed financial advisor or to post as the representative of a financial advisor. It is not permitted for Australian Property Forum members to ask for or offer specific buy, sell or hold recommendations on particular stocks, as a response to a request of this nature may be considered the provision of financial advice.

Views expressed on this forum are not representative of the forum owners. The forum owners are not liable or responsible for comments posted. Information posted does not constitute financial or legal advice. The forum owners accept no liability for information posted, nor for consequences of actions taken on the basis of that information. By visiting or using this forum, members and guests agree to be bound by the Zetaboards Terms of Use.

This site may contain copyright material (i.e. attributed snippets from online news reports), the use of which has not always been specifically authorized by the copyright owner. Such content is posted to advance understanding of environmental, political, human rights, economic, democratic, scientific, and social justice issues. This constitutes 'fair use' of such copyright material as provided for in section 107 of US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed for research and educational purposes only. If you wish to use this material for purposes that go beyond 'fair use', you must obtain permission from the copyright owner. Such material is credited to the true owner or licensee. We will remove from the forum any such material upon the request of the owners of the copyright of said material, as we claim no credit for such material.

For more information go to Limitations on Exclusive Rights: Fair Use

Privacy Policy: Australian Property Forum uses third party advertising companies to serve ads when you visit our site. These third party advertising companies may collect and use information about your visits to Australian Property Forum as well as other web sites in order to provide advertisements about goods and services of interest to you. If you would like more information about this practice and to know your choices about not having this information used by these companies, click here: Google Advertising Privacy FAQ

Australian Property Forum is hosted by Zetaboards. Please refer also to the Zetaboards Privacy Policy