Abeler et al. find that in a real effort task, effort provision and earnings are higher if the expectations-based reference point is high (HI treatment) compared to low (LO treatment).

Power Analysis

The original p-value is reported as p<0.05; we re-estimated the regression based on the posted data to get the exact p-value which is 0.046 based on a t-test of the coefficient (regression coefficient for “1 if HI treatment” dummy variable in the first regression in Table 1, an OLS model with the accumulated earnings in the real effort task as the dependent variable, p.478): “The treatment difference in effort provision is significant in an OLS regression where we compare effort in HI to effort in LO. We regress the accumulated earnings at which a subject stopped on a treatment dummy (see Table 1, column 1).”

The original sample size is 120 participants (60 in treatment T1 and 60 in treatment T2). To achieve 90% power the required sample size is 317 participants.

Sample

The sample for replication consists of 318 students at the University of Innsbruck in Austria, 159 subjects per each of the two treatments. As in the original study, we will exclude students with economics as a major and solely recruit subjects who had participated in no or only a few previous experiments.

Materials

We use the material of the original experiment (programmed in z-Tree) along with the original German instructions. The program is available on the journal’s webpage, while the German instructions have been kindly provided by the authors.

Procedure

We follow the procedure of the original article, with only slight but unavoidable deviations as outlined below. The following summary of the experimental procedure is therefore based on the section “I. Design” (pp. 473–475) in the original study.

The work task in this experiment consists of counting the number of zeros in tables containing randomly ordered zeros and ones. The experiment involves two stages. Prior to the first stage, subjects read the instructions (for the first stage only) and answer control questions. During the first stage, subjects have four minutes to count as many tables as possible. Thereby, the first stage serves to familiarize subjects with the task, permitting subjects to have a good understanding of how difficult the task is and how much they could earn in a given time before they know about the amount of the fixed payment (which is revealed only prior to the second stage).

After the first stage, subjects read the instructions for the second (and main) stage. Contrary to the first stage, subjects now can decide themselves how much and for how long they want to work. As soon as they push the “stop”-button on the screen, they get immediately paid and can leave. We employ the same procedure as in the original study and let subjects arrive for the experiment one at a time. Individual starting times are at least 20 minutes apart. Upon arrival, subjects are guided to one of three essentially identical, neutral rooms. Instructions and payments are also administered in their rooms.

Moreover, subjects do not earn their accumulated piece rate from the second stage for sure as they either receive a fixed rate or the accumulated piece rate earnings with equal probability. Which payment subjects receive is determined only after they have decided when to stop working. The two treatments HI and LOW only differ in the amount of the fixed payment (€7 in the HI treatment, €3 in the LOW treatment) which determines the rational expectation-based references. Subjects are randomly assigned to one of the two treatments.

After completing both stages, subjects will be privately paid in cash based on the same incentives and using the same show-up fee (€5) as in the original study (average earnings were €8.70 per subject in the original study).

Analysis Plan

The analysis will be performed exactly as in the original article. That is, differences in effort provision between the two treatments HI and LO are estimated by an OLS regression with the accumulated earnings at which a subject stopped as the dependent and a treatment dummy as independent variable.

Differences from Original Study

The replication procedure is identical to that of the original study, with the only unavoidable deviation that the replication will be performed at the University of Innsbruck, Austria, in 2015, on students from the University of Innsbruck, whereas the original data was gathered at the University of Bonn, Germany, in 2007, on students from the University of Bonn. However, as in the original study, only subjects who did not have economics as a major and who had participated in no or only a few previous experiments are invited. The experiment will be conducted in German as in the original study.

While the original study also looked at several control treatments, the focus of the replication is only on the earnings difference between the HI and the LO treatment.