This three-part series is designed to take an objective look at the data that we get from average draft position (ADP) reports, as the introductory post explains. Fantasy baseball draft research is only as good as the data that exists and the current system for generating ADP data from sites like MockDraftCentral is broken and in need of a new direction.
So, we know that the data that these websites produce is rather poor but why does it turn out that way and can we possibly fix it? Well, there are many problems that occur when a mock draft website offers rankings to these practicing drafters to base their selections off of. Psychologically, it causes a cognitive bias within the drafters at the site. It closely resembles an example the observer-expectancy effect in which the mock draft site is forcing their bias onto the participants.
The observer-expectancy effect is a form of reactivity, in which a researcher’s cognitive bias causes them to subconsciously influence the participants of an experiment. It is a significant threat to a study’s internal validity…
While I won’t bore you with any more psychology, the basic idea is that we should consider the host of a mock draft website to be a researchers and the mock drafters to be participants in the research. What results from this collaboration between researchers and participants should be reliable data in which we can determine where players are being drafted on average by the general fantasy baseball public. However, that is not what results from this collaboration unfortunately. Because of the obvious bias that the researchers pass on to the participants here, we are left with data that is not valid at all. The website asks the mock drafters to participate in a draft and then they give them a ranked list to choose from, which should obviously be a no-no as it subconsciously influences the participants.
The data that comes from these mock drafts is bad data since it is just a reflection of the site rankings themselves and this observer-expectancy effect is an explanation of why that is. If we truly want to know what the average public thinks about each player, we cannot continue with this current system that generates our ADP data. A new system must be created to reduce this blatant bias that exists in current mock drafts… But, what would that system look like?
