Assessing Markers of Reproducibility and Transparency in Smoking Behaviour Change Intervention Evaluations

Table 1

Measured variables and operationalization.

Variables	Coder questions	Response options
Article characteristics
Coder instructions: to identify journal impact factors use the Thomson Reuters Journal Citation Reports (https://library-guides.ucl.ac.uk/az.php?q=journal%20citation%20reports). For country, check the institutional affiliation of the corresponding author. If there are multiple corresponding authors, choose the first. If no corresponding author is identified, choose the first. If there are multiple affiliations for the selected author, choose the first.
Journal impact factor 2018	What is the 2018 journal impact factor?
Country	Which country is the corresponding author based in according to their affiliation?	[list countries]/unclear/other
Smoking outcome behaviour	Already extracted as part of the Human Behaviour-Change Project	Smoking cessation/smoking reduction/second-hand smoking
Behaviour change techniques (BCTs) in all study groups	Already extracted as part of the Human Behaviour-Change Project	93 BCTs of the Behaviour Change Techniques Taxonomy v1 [30] plus the addition of BCT 4.5 “advise to change behaviour”
Reproducibility
Preregistration
Definitions: “preregistration” refers to the specification of important aspects of the study (typically hypotheses, methods, and/or analysis plan) prior to commencement of the study.; Coder instructions: check specific sections in the paper where these files might be located, e.g., supplementary materials, appendices, author notes, methods, and results sections. Search for “registration”.
Preregistration statement	Does the article state whether or not the study (or some aspect of the study) was preregistered?	Yes—the statement says that there was a preregistration/yes—the statement says that there was no preregistration/no—there is no preregistration statement/other
Preregistration method	Where does the article indicate the preregistration is located?	Open Science Framework/AsPredicted/ClinicalTrials.gov/AEA trial registry/EGAP registry/tegistered report/other
Preregistration accessible	Can you access and open the preregistration?	Yes/no/other
Preregistration content	What aspects of the study appear to be preregistered? (select all that apply)	Hypotheses; Methods; Analysis plan; Other
Protocol sharing
Definition: “protocol” refers to a document containing details about the study design, methods, and analysis plan. It may or may not be preregistered.; Coder instructions: search the article for the phrase “protocol” and assess whether a link is provided to a protocol document.
Protocol availability	Does the article link to an accessible protocol?	Yes/no/other
Protocol content	What aspects of the study appear to be included in the protocol? (select all that apply)	Hypotheses; Methods; Analysis plan; Other
Data sharing
Definitions: “data” refers to recorded information that supports the analyses reported in the article. A “data availability statement” can be as simple as a url link to a data file or as complex as a written explanation as to why data cannot be shared.; Coder instructions: check the article for a data availability statement/link. They are often located in the “supplementary material,” “acknowledgements,” “author notes,” “methods,” or “results” sections. Search the article for the text “data availab” (to cover “data availability” and “data available”).
Data availability statement	Does the article state whether or not data are available?	Yes—the statement says that the data (or some of the data) are available/yes—the statement says that the data are not available/no—there is no data availability statement/other
Data sharing method	How does the statement indicate the data are available?	Upon request from the authors/personal or institution website/an online, third-party repository (e.g., OSF and FigShare)/supplementary materials hosted by the journal/other
Data accessibility	Can you access, download, and open the data files?	Yes/no/other
Data documentation	Are the data files clearly documented?	Yes/no/other
Data content	Do the data files appear to contain all of the raw data necessary to reproduce the reported findings?	Yes/no/unclear/other
Material sharing
Definitions: “material” refers to any study items that would be needed to repeat the study, such as stimuli, survey instruments, and computer code/software used for data collection, presentation stimuli, or running experiments.
Material availability statement	Does the article state whether or not materials are available?	Yes—the statement says that the materials (or some of the materials) are available/yes—the statement says that the materials are not available/no—there is no materials availability statement/other
Material sharing method	According to the statement, how are the materials accessible?	Upon request from the authors/personal or institution website/an online, third-party repository (e.g., OSF and FigShare)/supplementary materials hosted by the journal/other
Material accessibility	Can you access, download, and open the material files?	Yes/no/other
Analysis script sharing
Definition: “analysis scripts” refers to the specification of data preparation and analysis steps in the form of highly detailed step-by-step instructions for using point-and-click software, analysis code (e.g., R), or syntax (e.g., from SPSS).; Coder instructions: check the article for an analysis script availability statement/link. They are often located in the “supplementary material,” “acknowledgements,” “author notes,” “methods,” or “results” sections. Search for the text “analysis script” and “analysis code”.
Analysis script availability statement	Does the article state whether or not analysis scripts are available?	Yes—the statement says that the analysis scripts (or some of the analysis scripts) are available/yes—the statement says that the analysis scripts are not available/no—there is no analysis script availability statement
Analysis script sharing method	According to the statement, how are the analysis scripts accessible?	Upon request from the authors/personal or institution website/an online, third-party repository (e.g., OSF and FigShare)/supplementary materials hosted by the journal/other
Analysis script accessibility	Can you access, download, and open the analysis script files?	Yes/no/other
Replication
Definitions: “replication” refers to the repetition of a previous study’s methods in order to ascertain whether similar findings can be obtained.; Coder instructions: search the abstract and introduction for the phrase “replicat” (to cover “replication,” “replicates,” etc.). Confirm the authors are using the phrase with the definition provided above.
Replication statement	Does the article claim to report a replication study?	The article claims to report a replication study (or studies)/there is no clear statement that the article reports a replication study (or studies)/other
Open access
Coder instructions: to establish the open access status of the article: Go to https://openaccessbutton.org/ and enter the article’s doi (e.g., “10.1371/journal.pcbi.1004574”) if available (if not, enter the article title). If a link is provided, check that you can access the article at the link. If the article is accessible, answer “yes.” If the article is not accessible at the provided link, or no link is provided, answer “no.”
Open access status	Is the article open access?	Yes—found via open access button/yes—found via other means/no—could not access article other than through paywall/other
Transparency
Funding
Coder instructions: funding is usually reported in a specific section, e.g., “author information” or “funding statement.” Search the article for the phrase “funding”. If you are unsure whether an organisation is a tobacco company, pharmaceutical company, other private company, or public organisation, Google the organisation name and code accordingly. If it is unclear to you whether the funding is private or public, choose the “other” option and enter “unclear”.
Funding statement	Does the article include a statement indicating whether there were funding sources?	Yes—the statement says that there was funding from a tobacco company (e.g., Phillip Morris, British American Tobacco, China Tobacco, and Imperial Brands)/yes—funding from a pharmaceutical company (e.g., Pfizer and GSK)/yes—funding from another private company/yes—funding from a public organisation (e.g., National Institute of Health Research)/yes—the statement says that there was no funding provided/no—there is no funding statement/unclear/other
Conflict of interest
Coder instructions: conflicts of interest are usually reported in a specific section, e.g., “author information” or “conflict of interest statement.” Search the article for the phrases “conflict of interest” and/or “competing interest”. If you are unsure whether an organisation is a tobacco company, pharmaceutical company, other private company, or public organisation, Google the organisation name and code accordingly. If it is unclear to you whether the funding is private or public, choose the “other” option and enter “unclear”.
Conflict of interest statement	Does the article include a statement indicating whether there were any conflicts of interest?	Yes—the statement says that there was a conflict of interest from a tobacco company/yes—conflict of interest from a pharmaceutical company/yes—conflict of interest from another private company/yes—conflict of interest from a public organisation (e.g., National Institute of Health Research)/yes—the statement says that there is no conflict of interest/no—there is no conflict of interest statement/other

If a response marked with an asterisk is selected, the coder is asked to provide more detail in a free text response box. Note: identified measured variables have been adapted from a previous study assessing the transparency and reproducibility in psychological sciences [19].