babble-intro-img
babble is rabble.ca's discussion board but it's much more than that: it's an online community for folks who just won't shut up. It's a place to tell each other — and the world — what's up with our work and campaigns.

reliability and validity: What's the difference

janfromthebruce
Online
Joined: Apr 24 2007
 

Comments

janfromthebruce
Online
Joined: Apr 24 2007
Earlier this am I was working on posting a response in the thread concerning vapidity.
Got called away and when I went to post, the thread was closed.

The debate had turned into the reliability and validity of measurement and so this was the post I wanted to put in for what it is worth.

Let's look at reliability and later the validity of the "measures" used to ascertain both in proving which is more reliable and valid measure of vapid.

quote:If you google "she is vapid" and "he is vapid"

AND
quote:'he is vapid' produces 213,000 more hits than for 'she is vapid'.

Really folks this appears to be the argument.

Reliability and Validity: what's the Difference

quote:Reliability is the consistency of your measurement, or the degree to which an instrument measures the same way each time it is used under the same condition with the same subjects,
and is an estimate. There are two ways that reliability is usually estimated: test/retest and internal consistency.

Coming from a stats point of view, when using and not using " marks around words "he is vapid" or "she is vapid" in comparison to using no quotation marks actually shows "a lack of internal consistency." In other words, the instrument of measure "the use of quotation marks" is NOT reliable measure of these concepts, as removing the quote marks shows a complete different "estimate."

Concerning validity, as suggested in the reference page, "involves the degree to which one is measuring what one is supposed to, more simply, the accuracy of the measurement. I believe the the combined words have validity in the assertains stated here: he is vapid and she is vapid, but the internal consistency is lacking because inserting quotes or no quote marks should show internal consistency and don't.

Unionist you were trying to assert that only this reliability measure was valid, but Test/Retest, is the more conservative method to estimate reliability. Essentially what Remind showed was that your "measurement" lacked "internal consistency." She used a different grouping (no quote marks) to measure the same concepts of he is vapid and she is vapid. The difference in numbers of responses in the Google search for both genders did not generate a high correlation and thus lacks reliability.

I hope that made sense, and just wanted to move from a personal debate.


oldgoat
Offline
Joined: Jul 27 2001
It's been almost 30 years since I last took stats and research methodology courses. Interesting how little of it I remember, but I think I did retain that concept.

Michelle
Offline
Joined: May 10 2001
Since this seems to be more about research methods than a discussion about rabble content, should we move this to humanities and science?

oldgoat
Offline
Joined: Jul 27 2001
Ok, but if she starts talking about chi-square distributions I'm outta here.

Caissa
Offline
Joined: Jun 14 2006
Michelle wrote:Since this seems to be more about research methods than a discussion about rabble content, should we move this to humanities and science?

That would be valid. [img]wink.gif" border="0[/img]


M. Spector
Offline
Joined: Feb 19 2005
The difference between Googling "she is vapid" (with quotation marks) and she is vapid (without quotation marks) as a means of determining how often female entities are called vapid on the interweb is about the same as the difference between using an anemometer and a measuring cup to measure wind speed. One gives results that have a certain degree of both reliability and validity, while the other gives results that have neither validity nor reliability in any degree at all.

The alleged difference between reliability and validity therefore is nihil ad rem.


oldgoat
Offline
Joined: Jul 27 2001
quote:Originally posted by Caissa:

That would be valid. [img]wink.gif" border="0[/img]

She's reliable.


RosaL
Offline
Joined: Mar 4 2007
quote:Originally posted by M. Spector:
The difference between Googling "she is vapid" (with quotation marks) and she is vapid (without quotation marks) as a means of determining how often female entities are called vapid on the interweb is about the same as the difference between using an anemometer and a measuring cup to measure wind speed. One gives results that have a certain degree of both reliability and validity, while the other gives results that have neither validity nor reliability in any degree at all.

The alleged difference between reliability and validity therefore is nihil ad rem.

I have to agree with that. I also think Sarah Palin is vapid.


janfromthebruce
Online
Joined: Apr 24 2007
quote:Originally posted by oldgoat:
Ok, but if she starts talking about chi-square distributions I'm outta here.

Ok, so now onto chi-square distributions.

For those who are just oldgoats and really challenged, the internet provides a free Chi Square Calculator for all your computing enjoyment.

Once you get yours, we can all begin.... cause I am so vapid, she said [img]biggrin.gif" border="0[/img] [img]biggrin.gif" border="0[/img] [img]biggrin.gif" border="0[/img]


oldgoat
Offline
Joined: Jul 27 2001
Golly. And at first you seemed so nice.

M. Spector
Offline
Joined: Feb 19 2005
quote:Originally posted by RosaL:
I have to agree with that. I also think Sarah Palin is vapid.
I don't agree with that. Some of her statements are vapid, but I don't think that adjective applies to her as a person. It means dull, or lacking in liveliness.

P.S. That's another reason why Googling "she is vapid" is beside the point. The original reference to which that thread pointed was a reference to Alice Klein's "rhetorical vapidity", which is very different from saying Klein herself is "vapid".

That's why I did a Google search on "rhetorical vapidity" (with the quotes, natch) and showed that, on the internet at least, that phrase was not an anti-female trope.

[ 27 October 2008: Message edited by: M. Spector ]


RosaL
Offline
Joined: Mar 4 2007
quote:Originally posted by M. Spector:
I don't agree with that. Some of her statements are vapid, but I don't think that adjective applies to her as a person. It means dull, or lacking in liveliness.

All my life I'v been using it as a synonym for "superficial, empty". Incorrectly, however, as it now appears. [img]confused.gif" border="0[/img]


M. Spector
Offline
Joined: Feb 19 2005
Heh, heh.

You might also want to check out the difference between "immanent" and "imminent". [img]wink.gif" border="0[/img]


janfromthebruce
Online
Joined: Apr 24 2007
quote:Originally posted by oldgoat:
Golly. And at first you seemed so nice.

I still am oldgoat and so are you. [img]cool.gif" border="0[/img]


janfromthebruce
Online
Joined: Apr 24 2007
quote:Originally posted by M. Spector:
I don't agree with that. Some of her statements are vapid, but I don't think that adjective applies to her as a person. It means dull, or lacking in liveliness.

P.S. That's another reason why Googling "she is vapid" is beside the point. The original reference to which that thread pointed was a reference to Alice Klein's "rhetorical vapidity", which is very different from saying Klein herself is "vapid".

That's why I did a Google search on "rhetorical vapidity" (with the quotes, natch) and showed that, on the internet at least, that phrase was not an anti-female trope.

[ 27 October 2008: Message edited by: M. Spector ]

And I googled rhetorical vapiditywithout quotes to see if it had internal consistency, and guess what it did!


RosaL
Offline
Joined: Mar 4 2007
quote:Originally posted by M. Spector:
Heh, heh.

You might also want to check out the difference between "immanent" and "imminent". [img]wink.gif" border="0[/img]

Well, I do know that - it was just a spelling error. Really [img]wink.gif" border="0[/img]


oldgoat
Offline
Joined: Jul 27 2001
quote:Originally posted by janfromthebruce:

I still am oldgoat and so are you. [img]cool.gif" border="0[/img]


Well yes, very true. Just a brief note of explanation and I'll no longer bother the mathematically competent. In grade school, I was lousy at math, and in high school it became a real full blown phobia. Combined with other things going on in my life, it caused me to never actually pass grade 10. When I finally made it to university years later as a mature student I found I needed to take stats if I wanted to get an MSW, but I might as well have tried to learn to fly. Oddly enough I managed research methodology ok, which uses math differently.

I ended up doing the same sort of stuff anyway, but through a more roundabout route and with a history degree.


So carry on with your serious discussion.
[img]smile.gif" border="0[/img]

[ 27 October 2008: Message edited by: oldgoat ]


M. Spector
Offline
Joined: Feb 19 2005
quote:Originally posted by janfromthebruce:
And I googled rhetorical vapiditywithout quotes to see if it had internal consistency, and guess what it did!
My guess would be that it returned a whole bunch of web pages that had nothing whatsoever to do with the phrase "rhetorical vapidity" and therefore produced no relevant statistics at all.

And I just did that search to confirm that my guess was spot on.


Unionist
Online
Joined: Dec 11 2005
quote:Originally posted by janfromthebruce:
Unionist you were trying to assert that only this reliability measure was valid, but [b]Test/Retest, is the more conservative method to estimate reliability. Essentially what Remind showed was that your "measurement" lacked "internal consistency." She used a different grouping (no quote marks) to measure the same concepts of he is vapid and she is vapid. The difference in numbers of responses in the Google search for both genders did not generate a high correlation and thus lacks reliability.

I hope that made sense, and just wanted to move from a personal debate.[/b]

Hi jftb, just noticed your post for the first time.

You're mistaken. Remind's search was meaningless and measured nothing. Why is this not obvious??


kropotkin1951
Offline
Joined: Jun 6 2002
I think that the BNL hypothesis that a K car was reliable has been tested and retested. [img]wink.gif" border="0[/img]

janfromthebruce
Online
Joined: Apr 24 2007
quote:Originally posted by oldgoat:


Well yes, very true. Just a brief note of explanation and I'll no longer bother the mathematically competent. In grade school, I was lousy at math, and in high school it became a real full blown phobia. Combined with other things going on in my life, it caused me to never actually pass grade 10. When I finally made it to university years later as a mature student I found I needed to take stats if I wanted to get an MSW, but I might as well have tried to learn to fly. Oddly enough I managed research methodology ok, which uses math differently.

I ended up doing the same sort of stuff anyway, but through a more roundabout route and with a history degree.


So carry on with your serious discussion.
[img]smile.gif" border="0[/img]

[ 27 October 2008: Message edited by: oldgoat ]

Hey, oldgoat I know about that request to get into an MSW program. Classical stats is well classical, and that math stuff is trying. In my own MSW I completely avoided quantitative and used qualitative design and avoided that stuff. Good that things changed.


Unionist
Online
Joined: Dec 11 2005
I've always maintained, if you can't spend quality time with your family, settle for quantity time.

Michelle
Offline
Joined: May 10 2001
oldgoat, same here re: math. I dropped math after grade 10 math (which I barely squeaked through).

Then I took a first year economics course in university which required some calculus. I got 92%.

I think my math phobia was just that - a phobia. I'd probably be good at it if I did it now, but I've never gotten around to trying.

[ 27 October 2008: Message edited by: Michelle ]


Boom Boom
Offline
Joined: Dec 29 2004
I have quite a serious hearing loss, not quite made up for with a hearing aid, so listening to theory in math class in high school was a struggle, and I didn't do all that well. Then, I got accepted into college as a mature student a few years after finishing the four year tech program in high school, and things improved dramatically - smaller classes and more of a serious attitude from fellow students. Along with the required courses in Communication Arts, I did Micro and Macro Economics, and that was an eye opener. After I finished college, I worked as a program officer for the Local Initiatives Program in Ottawa (early 1970s) for a few years, saved big bux, and went to university to do a bachelor's in Experimental Psychology, which was mostly advanced statistical analysis. I did quite well, so having had difficulty with math in high school weasn't a harbinger of anything at all. It wasn't until I got out of high school that I was able to really get anywhere. Later on I did a Master's in a completely unrelated field. I have two older brothers who had careers in management and accounting, and I think I could have done the same.

Fidel
Offline
Joined: Apr 29 2004
quote:Originally posted by Michelle:

Then I took a first year economics course in university which required some calculus. I got 92%.

Holy crap! That's a pretty high mark for an upper level math course like calculus. That's not easy stuff.


Michelle
Offline
Joined: May 10 2001
No no. I mean, I got 92% in my economics course. I've never taken calculus in university or high school. But once I was doing a bit of it practically, in my economics course, it came easily to me.

In fact, I found that economics course so interesting that I wanted to major in it. But at Queen's, where I was going at the time, you have to take first year calculus in order to complete the Economics degree - it was a required course. Terribly disappointing to me, but I knew there was no way I could pass a calculus course when I had absolutely no math background to speak of - I can barely do basic algebra.

So I majored in Philosophy instead. [img]biggrin.gif" border="0[/img]

[ 27 October 2008: Message edited by: Michelle ]


Fidel
Offline
Joined: Apr 29 2004
quote:Originally posted by Boom Boom:
and went to university to do a bachelor's in Experimental Psychology, which was mostly advanced statistical analysis. I did quite well,

Holy frijole! I knew babblers weren't slow, but I'm beginning to realize I'm keepin some pretty fast company here... [img]smile.gif" border="0[/img]


Fidel
Offline
Joined: Apr 29 2004
quote:Originally posted by Michelle:
I can barely do basic algebra.

So I majored in Philosophy instead. [img]biggrin.gif" border="0[/img]

[ 27 October 2008: Message edited by: Michelle ]

I think if you can master what few rules of highschool algebra there are, that's a big part of calculus nailed down right there. I flunked grade eleven general level math when I was young, and so that, im my mind, reinforced the idea that upper level math was impossible for me. Like you and Boom Boom, I was wrong about that, too.


Boom Boom
Offline
Joined: Dec 29 2004
I gravitated to experimental psych partly because I could control my surroundings - I'm very hard of hearing - and doing experiments and then statistical analysis of these was an environment that allowed me to work at my own speed. [img]smile.gif" border="0[/img]

Boom Boom
Offline
Joined: Dec 29 2004
I feel a 'Nature versus Nurture' debate coming on... [img]biggrin.gif" border="0[/img]

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Login or register to post comments