The Volebny Kompas datasets (Mintal et al. 2024), tested on Figshare item 24949470¶
Paper. Mintal, Borseková, Cicchi, Müller, Vancel, Šimková and Deegan-Krause, "The Volebny Kompas Datasets on Slovak Voter and Party Positions", Scientific Data 11, 2024, doi:10.1038/s41597-024-03777-0.
Dataset. Figshare item 24949470, version 1, doi:10.6084/m9.figshare.24949470.v1, published 2024-07-09. The paper cites the item without a version (10.6084/m9.figshare.24949470); version 1 is the only version the archive lists, so it is the most recent published version and the one pinned here. The files are streamed from Figshare into memory; the cell below lists them with their MD5 from the archive's file list.
Claims. claims/figshare-24949470/10.1038-s41597-024-03777-0.yaml, written from the version of
record (PMC11374779). The paper's figures 2, 3 and 7, which two claims compare with, were read from
the published version of record by a separate agent that had not seen the computed placements, by pixel
position against the figures' axes; its readings, with the pixel positions of the axes they rest on,
are typed into this notebook. The deposited R code, Volebny_Kompas_code.R, is in the same item and is read
in the data check; where it fixes a detail the claims file leaves open, the section says so.
| Claim | Tested here | Outcome (see the record) |
|---|---|---|
| user-responses-count | yes | |
| collection-window | yes | |
| party-positions-structure | yes | |
| eleven-parties | yes | |
| party-self-placements | yes | |
| coder-agreement-kappa | yes | |
| uni-and-landscape-placements | yes, with the paper's Figs. 2 and 3 | |
| party-clustering | yes, with the paper's Fig. 7 | |
| table2-gender, table2-age, table2-education | yes | |
| age-education-inconsistencies | yes | |
| s2-s3-wide-range, s4-s5-consensus | yes | |
| efa-eleven-factors, efa-loadings-consistent | yes | |
| selected-parties-top-vote | blocked: official results not available in a pinned form | |
| website-views | blocked: the platform's usage records are not public |
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from scipy.cluster import hierarchy
from scipy.optimize import linear_sum_assignment
from scipy.spatial.distance import pdist
import vk
pd.set_option("display.width", 200)
pd.set_option("display.max_columns", 60)
files = vk.file_list()
meta = files.attrs["meta"]
print(meta["title"], "| version", meta["version"], "|", meta["doi"], "| published", meta["published_date"])
print(files.to_string(index=False))
The Volebny Kompas Datasets on Slovak Voter and Party Positions | version 1 | 10.6084/m9.figshare.24949470.v1 | published 2024-07-09T15:44:00Z
id name size computed_md5
47522423 Volebny_Kompas_code.R 15560 d6bf14c02c6eea379a259062e5c66a53
47522426 Data Dictionary.txt 9365 6e33ae40d9bee69a5652a018d137127b
47522429 README.txt 4521 35c77c94b05a98e55b16cdb12c265535
47522432 Supplementary Table 1.xlsx 12337 7e50664e0e2a70ee932daf61302e69b3
47522444 data_without_email.csv 32279707 b0cf874fdee96902d79a4f47664d324d
47522438 VAA_party_positions coded.xlsx 212812 252fd91c7f7bb0dc959c7f21f60f22fa
47522435 recoded_party_positions.csv 1661 1bc5b17a5ebb7a043579bd0e7ef80185
47522441 census_final.csv 5842 ecfce47c59fd35080d1bdda22a601afa
Data check¶
Everything the item holds is loaded and listed: the README, the data dictionary and the R code (text),
the user responses (CSV, ;-separated), every sheet of the two workbooks, the recoded party positions
and the census table. The checks of the description against the values follow the printout.
texts = {n: vk.read_bytes(vk.file_id(files, n)).decode("utf-8") for n in ["README.txt", "Data Dictionary.txt", "Volebny_Kompas_code.R"]}
for n, t in texts.items():
print(f"{n}: {len(t.splitlines())} lines")
raw, likert = vk.load_responses(files)
supp = vk.load_supplementary_table(files)
sheets, pp = vk.load_party_positions(files, supp)
rec = vk.load_recoded(files)
census = pd.read_csv(pd.io.common.BytesIO(vk.read_bytes(vk.file_id(files, "census_final.csv"))))
md5 = files.assign(streamed=files["id"].map(vk.STREAMED_MD5))
print("MD5 of the streamed bytes equals the archive's for every file:", bool((md5["streamed"] == md5["computed_md5"]).all()),
"| files streamed:", int(md5["streamed"].notna().sum()), "of", len(md5))
print("\nUser responses:", raw.shape, "columns:", list(raw.columns))
print("Values in S1 to S39 (as read):", pd.Series(raw[vk.STATEMENTS].to_numpy().ravel()).value_counts().to_dict())
for c in ["UN2", "UN3", "UN4"]:
v = raw[c]
print(f"{c}: {(v == '').sum()} empty, {v[v != ''].nunique()} distinct codes, e.g. {sorted(v[v != ''].unique())[:5]}")
print("UN5 to UN26 values:", sorted(set(raw[[f'UN{i}' for i in range(5, 27)]].to_numpy().ravel())))
print("Share of 1 in UN5 to UN26:", raw[[f'UN{i}' for i in range(5, 27)]].astype(int).mean().round(3).to_dict())
for c in ["GENDER", "AGE", "EDUCATION"]:
print(c, raw[c].value_counts().to_dict())
blank = raw[["GENDER", "AGE", "EDUCATION"]].eq("")
print("Rows with gender, age, education empty:", blank.sum().to_dict(), "| all three empty together:", int(blank.all(axis=1).sum()),
"| some but not all empty:", int((blank.any(axis=1) & ~blank.all(axis=1)).sum()))
email = raw["EMAIL_ID"]
print("EMAIL_ID: NA", int((email == "NA").sum()), "| given", int((email != "NA").sum()), "| distinct", email[email != "NA"].nunique(),
"| largest repeat", int(email[email != "NA"].value_counts().max()))
ts = pd.to_datetime(raw["TIME.STAMP"], format="%Y-%m-%d %H:%M:%S", errors="coerce")
print("TIME.STAMP: unparsed", int(ts.isna().sum()), "| first", ts.min(), "| last", ts.max())
print("Rows identical in every column to another row:", int(raw.duplicated(keep=False).sum()),
"| identical in TIME.STAMP and S1 to S39:", int(raw.duplicated(["TIME.STAMP"] + vk.STATEMENTS, keep=False).sum()))
README.txt: 109 lines Data Dictionary.txt: 212 lines Volebny_Kompas_code.R: 433 lines
MD5 of the streamed bytes equals the archive's for every file: True | files streamed: 8 of 8 User responses: (134699, 69) columns: ['TIME.STAMP', 'S1', 'S2', 'S3', 'S4', 'S5', 'S6', 'S7', 'S8', 'S9', 'S10', 'S11', 'S12', 'S13', 'S14', 'S15', 'S16', 'S17', 'S18', 'S19', 'S20', 'S21', 'S22', 'S23', 'S24', 'S25', 'S26', 'S27', 'S28', 'S29', 'S30', 'S31', 'S32', 'S33', 'S34', 'S35', 'S36', 'S37', 'S38', 'S39', 'UN2', 'UN3', 'UN4', 'UN5', 'UN6', 'UN7', 'UN8', 'UN9', 'UN10', 'UN11', 'UN12', 'UN13', 'UN14', 'UN15', 'UN16', 'UN17', 'UN18', 'UN19', 'UN20', 'UN21', 'UN22', 'UN23', 'UN24', 'UN25', 'UN26', 'GENDER', 'AGE', 'EDUCATION', 'EMAIL_ID']
Values in S1 to S39 (as read): {'100': 1524755, '75': 1214227, '0': 830460, '25': 788619, '50': 687277, 'NA': 183459, '-1': 24464}
UN2: 43651 empty, 39 distinct codes, e.g. ['1-1', '1-2', '1-3', '1-4', '1-5']
UN3: 48204 empty, 39 distinct codes, e.g. ['1-1', '1-2', '1-3', '1-4', '1-5']
UN4: 54332 empty, 39 distinct codes, e.g. ['1-1', '1-2', '1-3', '1-4', '1-5']
UN5 to UN26 values: ['0', '1']
Share of 1 in UN5 to UN26: {'UN5': 0.01, 'UN6': 0.2, 'UN7': 0.075, 'UN8': 0.085, 'UN9': 0.061, 'UN10': 0.513, 'UN11': 0.035, 'UN12': 0.344, 'UN13': 0.031, 'UN14': 0.05, 'UN15': 0.036, 'UN16': 0.212, 'UN17': 0.083, 'UN18': 0.349, 'UN19': 0.241, 'UN20': 0.33, 'UN21': 0.116, 'UN22': 0.581, 'UN23': 0.076, 'UN24': 0.408, 'UN25': 0.625, 'UN26': 0.518}
GENDER {'Male': 62240, 'Female': 45358, '': 25639, 'Other / Prefer not to say': 1462}
AGE {'25-34': 35277, '35-44': 28095, '': 25639, '45-54': 15275, '21-24': 12446, '18-20': 7138, '55-64': 6111, '65-74': 2731, 'Under 18': 1285, '75 or older': 702}
EDUCATION {'University degree/Academic higher education': 70150, 'Secondary education': 30998, '': 25639, 'Applied higher education': 2605, 'Vocational education': 2587, 'Primary education': 1686, 'No education': 1034}
Rows with gender, age, education empty: {'GENDER': 25639, 'AGE': 25639, 'EDUCATION': 25639} | all three empty together: 25639 | some but not all empty: 0
EMAIL_ID: NA 102690 | given 32009 | distinct 30775 | largest repeat 8
TIME.STAMP: unparsed 0 | first 2023-09-05 11:26:31 | last 2023-10-09 08:57:38
Rows identical in every column to another row: 0 | identical in TIME.STAMP and S1 to S39: 0
print("Workbook sheets:", {k: v.shape for k, v in sheets.items()})
cs = sheets["Codesheet"]
print("Codesheet columns:", list(cs.columns))
print("Rows naming a party:", len(pp), "| rows below them (a legend):", len(cs) - len(pp))
print(cs.loc[cs["Party"].isna(), "Unique ID"].dropna().tolist())
for c in ["Self-placement position", "Self-placement source", "Expert position I", "Expert position II", "Coders Match", "Final position", "Final source"]:
print(f"{c}: {pp[c].value_counts(dropna=False).to_dict()}")
print("Non-empty text columns:", {c: int(pp[c].notna().sum()) for c in ["Self-placement snippet", "Self-placement link", "Expert snippet I", "Expert link I",
"Expert snippet II", "Expert link II", "Final link", "Final snippet"]})
mism = pp[pp["statement"] != pp["Statement no."]]
print(f"\nRows whose 'Statement no.' differs from the number of their English wording in Supplementary Table 1: {len(mism)} of {len(pp)}")
print(mism.groupby(["Statement no.", "statement"]).size().rename("rows").reset_index().to_string(index=False))
print("Every row's wording matched a statement:", bool(pp["statement"].notna().all()))
print("\nSupplementary Table 1:", supp.shape, list(supp.columns))
print("Recoded party positions:", rec.shape, "values", sorted(set(rec.to_numpy().ravel())))
final = pp.pivot(index="Party", columns="statement", values="Final position").replace(vk.LIKERT).replace("No opinion", 50).astype(float)
by_no = pp.pivot(index="Party", columns="Statement no.", values="Final position").replace(vk.LIKERT).replace("No opinion", 50).astype(float)
names = {"OĽANO": "OLANO"}
r2 = rec.rename(index=names)
r2.columns = range(1, 40)
print("Recoded CSV equals the workbook's final positions (No opinion as 50), statements numbered by wording:",
f"{(r2 == final.loc[r2.index]).to_numpy().mean():.3f}", "| numbered by 'Statement no.':", f"{(r2 == by_no.loc[r2.index]).to_numpy().mean():.3f}")
print("\nCensus table:", census.shape, list(census.columns))
print(census.drop(columns="Total_Observations").nunique().to_dict())
Workbook sheets: {'Codesheet': (446, 21)}
Codesheet columns: ['Unique ID', 'Statement in English', 'Statement in Slovak', 'Statements in Hungarian', 'Statement no.', 'Party', 'Self-placement position', 'Self-placement source', 'Self-placement snippet', 'Self-placement link', 'Expert position I', 'Expert snippet I', 'Expert link I', 'Expert position II', 'Expert snippet II', 'Expert link II', 'Coders Match', 'Final position', 'Final link', 'Final snippet', 'Final source']
Rows naming a party: 429 | rows below them (a legend): 17
['Completely disagree', 'Tend to disagree', 'Neutral', 'Tend to agree', 'Completely agree', 'No opinion', 'Current (2023) parliamentary election manifesto', 'Party programme', 'Other programmatic and official party documentation', 'Interviews, press releases and social media communication from party leader or other leading candidates', 'Older election manifestos', 'Other sources', '*PS provided self-placing only via agree/disagree']
Self-placement position: {nan: 298, 'Tend to agree': 28, 'Completely disagree': 24, 'Completely agree': 23, 'agree': 19, 'Tend to disagree': 16, 'disagree': 13, 'Neutral': 8}
Self-placement source: {nan: 429}
Expert position I: {'No opinion': 123, 'Completely agree': 99, 'Completely disagree': 69, 'Tend to agree': 67, 'Tend to disagree': 53, 'Neutral': 18}
Expert position II: {'Completely agree': 138, 'No opinion': 105, 'Completely disagree': 71, 'Tend to disagree': 54, 'Tend to agree': 53, 'Neutral': 8}
Coders Match: {'Exact match': 178, 'no opinion&categorized': 99, 'Both no opinion': 65, 'Close match': 57, 'No match': 30}
Final position: {'Completely agree': 122, 'Completely disagree': 81, 'Tend to agree': 77, 'No opinion': 69, 'Tend to disagree': 64, 'Neutral': 16}
Final source: {'Interviews, press releases and social media communication from party leader or other leading candidates': 182, 'Current (2023) parliamentary election manifesto': 104, nan: 68, 'Other sources': 29, 'Other programmatic and official party documentation': 19, 'Party programme': 17, 'Older election manifestos': 10}
Non-empty text columns: {'Self-placement snippet': 117, 'Self-placement link': 1, 'Expert snippet I': 305, 'Expert link I': 306, 'Expert snippet II': 320, 'Expert link II': 323, 'Final link': 361, 'Final snippet': 361}
Rows whose 'Statement no.' differs from the number of their English wording in Supplementary Table 1: 99 of 429
Statement no. statement rows
16 19 11
17 20 11
18 21 11
19 22 11
20 23 11
21 24 11
22 16 11
23 17 11
24 18 11
Every row's wording matched a statement: True
Supplementary Table 1: (39, 6) ['Statement Number', 'Statement (English)', 'Uni-Dimension', 'Statement Uni-Dimension Polarity', 'Political Landscape Dimension(s)', 'Statement Political Landscape Dimension(s) Polarity']
Recoded party positions: (11, 39) values [np.int64(0), np.int64(25), np.int64(50), np.int64(75), np.int64(100)]
Recoded CSV equals the workbook's final positions (No opinion as 50), statements numbered by wording: 1.000 | numbered by 'Statement no.': 0.823
Census table: (124, 4) ['Gender', 'AgeCategory', 'EducationCategory', 'Total_Observations']
{'Gender': 2, 'AgeCategory': 9, 'EducationCategory': 7}
Checks of the description against the data:
- The MD5 of every streamed file equals the one the archive lists.
- The CSV has 134,699 rows and the 69 columns of the data dictionary, in its order. "No opinion" is the
string
NAin S1 to S39, not an empty string as the paper's Data cleaning says; unanswered statements are -1, as described. The S values are 0, 25, 50, 75 and 100 only. - UN2 to UN4 hold the three most salient statements as codes such as
6-1(the dictionary's two-digit identifiers, block and item), not as S numbers. UN5 to UN15 and UN16 to UN26 are 0/1, as described. Gender, age and education are empty together in the same rows; the dictionary gives no categories, and the values are those of the R code's validation rules. The age labels use a hyphen ("18-20") where the paper prints an en dash; they are the same labels. - Table 1 of the paper shows
2023-10-09 08:57:38as a sample timestamp: it is the first row of the file, which is sorted newest first, and the last date in it. The timestamps cover 5 September to 9 October 2023 (see collection-window). - EMAIL_ID repeats, as the paper says it may. The README says each row is "a unique respondent"; the paper says each row is a unique response. The repeats show that some respondents answered more than once.
- The party workbook has one sheet with 429 coded rows and a legend of the categories below them. Its "Statement no." column disagrees with the English wording for 99 rows: the rows worded as statements 16 to 24 are numbered 22, 23, 24, 16, 17, 18, 19, 20, 21 for every party. The recoded CSV follows the wording, and so do the paper's Figures 2 and 3 (see uni-and-landscape-placements), so the wording is taken as right and the number as wrong. All analyses here number statements by their wording.
- The data dictionary lists "Self-placement source" as a column; it is empty in all 429 rows.
- This is survey data with one timestamp per response and no trial structure, so the protocol's check that all streams share one clock does not apply.
The deposited R code¶
The lines of Volebny_Kompas_code.R that the analysis relies on, printed from the streamed file: the
straight-line rows, the removal of incomplete rows, "no opinion" as 50, the validation rules for gender, age
and education, the unique-email rows, the parallel analyses and factor analyses, the clustering of the recoded
party positions and the optional moratorium filter. The README's description of the recoded party positions
follows.
rlines = texts["Volebny_Kompas_code.R"].splitlines()
excerpts = [("# Identify and exclude rows where all values", 3), ("rows_with_minus_one <-", 8), ("# Replace NA with 50", 5),
("# Define validation rules considering NA as valid", 6), ("## FILTERING DATA WITH UNIQUE EMAIL ID", 11),
("cor_matrix <- cor(", 1), ("parallel_pearson <-", 1), ("fa_result_ml <-", 1), ("fa_result_pa <-", 1),
('data <- read.csv("recoded_party_positions.csv")', 1), ("dist_matrix <-", 1), ("hclust_result <-", 1),
("polychoric_matrix <- polychoric", 1), ("parallel_polychoric <-", 1), ("fa_result <- fa(polychoric_matrix", 1),
("## FILTER OUT DATA UNTIL ELECTION MORATORIUM", 18)]
for anchor, n in excerpts:
i = next(k for k, l in enumerate(rlines) if l.startswith(anchor))
for k in range(i, i + n):
print(f"{k + 1:4d} {rlines[k]}")
print(" ...")
print("lines naming both AGE and EDUCATION:", [l.strip() for l in rlines if "AGE" in l and "EDUCATION" in l])
print("\nREADME on the recoded party positions:")
print("\n".join(l for l in texts["README.txt"].splitlines() if "recoded them" in l))
37 # Identify and exclude rows where all values in columns S1 to S39 are the same
38 same_response_rows <- apply(data[, s1_to_s39_cols], 1, function(row) length(unique(row)) == 1)
39 cleansed_data <- data[!same_response_rows, ]
...
42 rows_with_minus_one <- apply(cleansed_data[, s1_to_s39_cols], 1, function(row) {
43 row_no_na <- row[!is.na(row)]
44 any(row_no_na == -1)
45 })
46
47 # Exclude rows with '-1' values
48 data_cleaned_corrected <- cleansed_data[!rows_with_minus_one, ]
49
...
56 # Replace NA with 50 for columns S1 to S39
57 data_cleaned_corrected[, s1_to_s39_cols] <- lapply(data_cleaned_corrected[, s1_to_s39_cols], function(col) {
58 col[is.na(col)] <- 50
59 return(col)
60 })
...
101 # Define validation rules considering NA as valid
102 rules <- validator(
103 GENDER %in% c("Male", "Female", "Other / Prefer not to say") | is.na(GENDER),
104 AGE %in% c("Under 18", "18-20", "21-24", "25-34", "35-44", "45-54", "55-64", "65-74", "75 or older") | is.na(AGE),
105 EDUCATION %in% c("No education", "Primary education", "Vocational education", "Secondary education", "Applied higher education", "University degree/Academic higher education") | is.na(EDUCATION)
106 )
...
133 ## FILTERING DATA WITH UNIQUE EMAIL ID ##
134
135 # Filter rows with unique EMAIL_ID, excluding NA values
136 unique_email_data_dplyr <- data_cleaned_corrected %>%
137 filter(!is.na(EMAIL_ID)) %>%
138 group_by(EMAIL_ID) %>%
139 filter(n() == 1) %>%
140 ungroup()
141
142 # Display the dimensions
143 dim(unique_email_data_dplyr)
...
204 cor_matrix <- cor(fa_data, use = "pairwise.complete.obs")
...
213 parallel_pearson <- fa.parallel(cor_matrix, n.obs = n_cases, fm = "ml", fa = "fa", n.iter = 1000)
...
221 fa_result_ml <- fa(cor_matrix, nfactors = n_factors, rotate = "promax", fm = "ml")
...
224 fa_result_pa <- fa(cor_matrix, nfactors = n_factors, rotate = "promax", fm = "pa")
...
285 data <- read.csv("recoded_party_positions.csv")
...
292 dist_matrix <- dist(data, method = "manhattan")
...
295 hclust_result <- hclust(dist_matrix, method = "average")
...
346 polychoric_matrix <- polychoric(fa_data)$rho
...
355 parallel_polychoric <- fa.parallel(polychoric_matrix, n.obs = n_cases, fm = "ml", fa = "fa", n.iter = 100)
...
360 fa_result <- fa(polychoric_matrix, nfactors = n_factors, fm = "ml")
...
370 ## FILTER OUT DATA UNTIL ELECTION MORATORIUM (OPTIONAL) = approx. 400 ##
371 ###start date was manually defined as the first response in the dataset (when the VAA was launched), with the end date manually set to the time of the last response before the end of the election moratorium on September 30th at 22:45
372
373
374 # Convert TIME.STAMP
375 data_cleaned_corrected$TIME.STAMP <- as.POSIXct(data_cleaned_corrected$TIME.STAMP,
376 format = "%Y-%m-%d %H:%M:%S",
377 tz = "UTC")
378
379 # Define start and end
380 start_datetime <- as.POSIXct("2023-09-05 11:26:31", format = "%Y-%m-%d %H:%M:%S", tz = "UTC")
381 end_datetime <- as.POSIXct("2023-09-30 22:42:30", format = "%Y-%m-%d %H:%M:%S", tz = "UTC")
382
383 # Filter rows
384 filtered_data <- data_cleaned_corrected %>%
385 filter(TIME.STAMP >= start_datetime & TIME.STAMP <= end_datetime)
386
387 # View
...
lines naming both AGE and EDUCATION: ['# Replace empty strings with NA for GENDER, AGE, and EDUCATION', 'selected_data <- data_cleaned_corrected %>% select(all_of(s1_to_s39_cols), GENDER, AGE, EDUCATION)']
README on the recoded party positions:
For easy of use we have taken out the final positions from the VAA_party_positions coded.xlsx and recoded them into 0,25,50,75,100 so the data corresponds with the format of the data_without_email.csv, additionally in this dataset we have also recoded "no opinion" as 50.
Row selections¶
Four sets of rows are used, as the claims file names them:
raw: all 134,699 rows;paper: the paper's cleaning (Technical Validation, Data cleaning): rows with -1 in any of S1 to S39 removed;code: the deposited R code's cleaning, which also removes rows that give the same answer to all 39 statements (same_response_rowsinVolebny_Kompas_code.R), a step the paper does not mention;code_email: the R code's rows for the factor analysis,coderows whose EMAIL_ID is given and occurs once.
sel = vk.selections(raw, likert)
incomplete = (likert == -1).any(axis=1)
same = likert.fillna(-999).nunique(axis=1) == 1
print({k: int(v.sum()) for k, v in sel.items()})
print("rows with -1:", int(incomplete.sum()), "| rows with one answer to all 39 (and no -1):", int((same & ~incomplete).sum()))
print("answers of those straight-line rows:", likert[same & ~incomplete].iloc[:, 0].fillna(-999).value_counts().rename({-999: "all NA"}).to_dict())
{'raw': 134699, 'paper': 133473, 'code': 133143, 'code_email': 29493}
rows with -1: 1226 | rows with one answer to all 39 (and no -1): 330
answers of those straight-line rows: {'all NA': 204, 100.0: 51, 50.0: 49, 0.0: 24, 75.0: 1, 25.0: 1}
user-responses-count¶
The User Responses Dataset holds 134,699 voter responses, each with answers to the 39 statements, along with demographics, the three most salient statements and the parties the respondent would and would never vote for, for most responses.
table1 = ["TIME.STAMP"] + vk.STATEMENTS + [f"UN{i}" for i in range(2, 27)] + ["GENDER", "AGE", "EDUCATION", "EMAIL_ID"]
print("rows of the raw file:", len(raw), "| expected 134,699 +- 13,470")
print("columns of Table 1 and the dictionary present:", all(c in raw.columns for c in table1), "| missing:", [c for c in table1 if c not in raw.columns])
print("rows after removing incomplete ones (paper):", int(sel["paper"].sum()), "| after the R code's cleaning:", int(sel["code"].sum()))
share = {"demographics given": (raw["GENDER"] != "").mean(), "a salient statement given": (raw["UN2"] != "").mean(),
"a party considered": (raw[[f'UN{i}' for i in range(5, 16)]].astype(int).sum(axis=1) > 0).mean(),
"a party never considered": (raw[[f'UN{i}' for i in range(16, 27)]].astype(int).sum(axis=1) > 0).mean()}
print({k: round(float(v), 3) for k, v in share.items()})
rows of the raw file: 134699 | expected 134,699 +- 13,470
columns of Table 1 and the dictionary present: True | missing: []
rows after removing incomplete ones (paper): 133473 | after the R code's cleaning: 133143
{'demographics given': 0.81, 'a salient statement given': 0.676, 'a party considered': 0.732, 'a party never considered': 0.769}
Result: reproduced. The raw file has 134,699 rows (expected 134,699) and every column of Table 1; 81.0% of rows give demographics and 67.6% a salient statement, so "for most responses" holds. 133,473 rows remain after removing incomplete ones.
collection-window¶
The responses were gathered during the 26 days leading up to and including election day, from the launch on 5 September 2023 to the election on 30 September 2023.
Readings from the criterion: (a) timestamps as given; (b) timestamps read as UTC and converted to Slovak time (Europe/Bratislava, CEST). The R code's optional filter ends at 2023-09-30 22:42:30, "the time of the last response before the end of the election moratorium", and is shown as well.
start, end = pd.Timestamp("2023-09-05"), pd.Timestamp("2023-10-01")
readings = {"(a) as given": ts, "(b) UTC to Slovak time": ts.dt.tz_localize("UTC").dt.tz_convert("Europe/Bratislava").dt.tz_localize(None)}
for name, t in readings.items():
out = (t < start) | (t >= end)
print(f"{name}: first {t.min()}, last {t.max()}; outside 5 to 30 Sep: {int(out.sum())} of {len(t)} ({out.mean():.2%});",
"dates outside:", t[out].dt.date.value_counts().sort_index().to_dict())
late = ts > pd.Timestamp("2023-09-30 22:42:30")
print("rows after the R code's moratorium end (2023-09-30 22:42:30, as given): raw", int(late.sum()),
"| data_cleaned_corrected (the R code's cleaning, to which the code applies the filter):", int((late & sel["code"]).sum()))
for name, t in readings.items():
print(f"days with responses inside the window, {name}:", t[(t >= start) & (t < end)].dt.date.nunique())
print("first three rows of the file:", raw["TIME.STAMP"].head(3).tolist(), "| first row is the latest timestamp:", bool(ts.iloc[0] == ts.max()),
"| file sorted newest first:", bool(ts.is_monotonic_decreasing))
daily = ts.dt.date.value_counts().sort_index()
fig, ax = plt.subplots(figsize=(9, 2.5))
ax.bar(pd.to_datetime(daily.index), daily.values)
ax.set_ylabel("responses per day")
ax.set_title(f"Responses per day by TIME.STAMP as given ({daily.min()} to {daily.max()} per day)")
plt.show()
(a) as given: first 2023-09-05 11:26:31, last 2023-10-09 08:57:38; outside 5 to 30 Sep: 432 of 134699 (0.32%); dates outside: {datetime.date(2023, 10, 1): 195, datetime.date(2023, 10, 2): 98, datetime.date(2023, 10, 3): 44, datetime.date(2023, 10, 4): 37, datetime.date(2023, 10, 5): 19, datetime.date(2023, 10, 6): 14, datetime.date(2023, 10, 7): 13, datetime.date(2023, 10, 8): 11, datetime.date(2023, 10, 9): 1}
(b) UTC to Slovak time: first 2023-09-05 13:26:31, last 2023-10-09 10:57:38; outside 5 to 30 Sep: 494 of 134699 (0.37%); dates outside: {datetime.date(2023, 10, 1): 246, datetime.date(2023, 10, 2): 99, datetime.date(2023, 10, 3): 53, datetime.date(2023, 10, 4): 37, datetime.date(2023, 10, 5): 19, datetime.date(2023, 10, 6): 13, datetime.date(2023, 10, 7): 12, datetime.date(2023, 10, 8): 13, datetime.date(2023, 10, 9): 2}
rows after the R code's moratorium end (2023-09-30 22:42:30, as given): raw 461 | data_cleaned_corrected (the R code's cleaning, to which the code applies the filter): 451
days with responses inside the window, (a) as given: 26 days with responses inside the window, (b) UTC to Slovak time: 26 first three rows of the file: ['2023-10-09 08:57:38', '2023-10-08 23:29:51', '2023-10-08 20:42:05'] | first row is the latest timestamp: True | file sorted newest first: True
Result: partially reproduced. Under both readings the timestamps start on 5 September 2023 and cover all 26 days up to election day, but 432 responses as given (0.32%), and 494 read as UTC (0.37%), fall from 1 to 9 October 2023, after election day, so the criterion, every timestamp inside the window, fails; 99.7% of rows are inside it. The file is sorted newest first, so its first row, the sample of Table 1 (2023-10-09 08:57:38), is the latest timestamp and one of the late rows. The R code's optional filter, ending at 2023-09-30 22:42:30, would drop 451 rows of data_cleaned_corrected, the rows it is applied to (461 of the raw rows). The code parses both the timestamps and the cut-off with tz = "UTC", so it compares the values as given, and its comment gives the moratorium's end as 22:45 with no time zone; this suggests the timestamps are local time (reading a) but does not settle it. The outcome is the same under both readings.
party-positions-structure¶
The Party Positions Dataset lists the 39 statements in Slovak, Hungarian and English and holds 429 party positions, both party self-placements and expert-coded positions, with sources, snippets and links.
langs = ["Statement in English", "Statement in Slovak", "Statements in Hungarian"]
per_statement = pp.groupby("statement")[langs].nunique()
print("coded rows (positions):", len(pp), "| statements:", pp["statement"].nunique(), "| by 'Statement no.':", pp["Statement no."].nunique())
print("every row has all three wordings:", bool(pp[langs].notna().all().all()),
"| distinct wordings per statement (max):", per_statement.max().to_dict())
cols = ["Self-placement position", "Expert position I", "Expert position II", "Final position", "Self-placement source",
"Self-placement snippet", "Self-placement link", "Expert snippet I", "Expert link I", "Final source", "Final snippet", "Final link"]
print({c: int(pp[c].notna().sum()) for c in cols})
coded rows (positions): 429 | statements: 39 | by 'Statement no.': 39
every row has all three wordings: True | distinct wordings per statement (max): {'Statement in English': 1, 'Statement in Slovak': 1, 'Statements in Hungarian': 1}
{'Self-placement position': 131, 'Expert position I': 429, 'Expert position II': 429, 'Final position': 429, 'Self-placement source': 0, 'Self-placement snippet': 117, 'Self-placement link': 1, 'Expert snippet I': 305, 'Expert link I': 306, 'Final source': 361, 'Final snippet': 361, 'Final link': 361}
Result: reproduced. 429 positions on 39 statements, each with its Slovak, Hungarian and English wording; there are self-placement, expert (two coders) and final positions, with snippet and link columns for each and a source column for the final positions. The self-placement source column is empty.
eleven-parties¶
Eleven parties are positioned on the 39 statements.
cover = pp.groupby("Party")["statement"].agg(["nunique", "size"])
print(cover.to_string())
print("parties:", pp["Party"].nunique(), "| each with 39 distinct statements:", bool((cover["nunique"] == 39).all()))
print("recoded CSV parties:", rec.index.tolist())
nunique size Party ALIANCIA 39 39 DEMOKRATI 39 39 HLAS-SD 39 39 KDH 39 39 OLANO 39 39 PS 39 39 Republika 39 39 SME Rodina 39 39 SMER-SD 39 39 SNS 39 39 SaS 39 39 parties: 11 | each with 39 distinct statements: True recoded CSV parties: ['HLAS-SD', 'PS', 'SMER-SD', 'SME Rodina', 'Republika', 'SaS', 'OĽANO', 'KDH', 'DEMOKRATI', 'SNS', 'ALIANCIA']
Result: reproduced. The workbook names 11 parties, each with a position on all 39 statements.
party-self-placements¶
Four of the eleven parties returned the self-placement questionnaire.
sp = pp.groupby("Party")["Self-placement position"].agg(lambda s: int(s.notna().sum()))
print(sp.sort_values(ascending=False).to_string())
print("parties with at least one self-placement:", int((sp > 0).sum()))
print("values used by PS:", pp.loc[pp["Party"] == "PS", "Self-placement position"].value_counts().to_dict())
Party
SaS 39
DEMOKRATI 38
PS 32
KDH 22
ALIANCIA 0
HLAS-SD 0
OLANO 0
Republika 0
SME Rodina 0
SMER-SD 0
SNS 0
parties with at least one self-placement: 4
values used by PS: {'agree': 19, 'disagree': 13}
Result: reproduced. Four parties have self-placements: SaS on 39 statements, DEMOKRATI on 38, PS on 32 and KDH on 22. PS answered only "agree" or "disagree", as the workbook's note says.
coder-agreement-kappa¶
Agreement between the two expert coders of party positions was high, with a weighted Cohen's kappa of 0.84.
Readings from the criterion: weights (a) linear or (b) quadratic; "No opinion" (i) pairs with No opinion from either coder left out, or (ii) scored as Neutral. The expected value is 0.84 +- 0.084.
cats = ["Completely disagree", "Tend to disagree", "Neutral", "Tend to agree", "Completely agree"]
a, b = pp["Expert position I"], pp["Expert position II"]
both = a.isin(cats) & b.isin(cats)
rows = []
for w in ["linear", "quadratic"]:
rows.append((w, "(i) No opinion left out", int(both.sum()), vk.weighted_kappa(a[both], b[both], cats, w)))
rows.append((w, "(ii) No opinion as Neutral", len(a), vk.weighted_kappa(a.replace("No opinion", "Neutral"), b.replace("No opinion", "Neutral"), cats, w)))
kap = pd.DataFrame(rows, columns=["weights", "No opinion", "pairs", "kappa"])
kap["within 0.84 +- 0.084"] = (kap["kappa"] - 0.84).abs() <= 0.084
print(kap.round(3).to_string(index=False))
print("unweighted agreement on the 266 pairs:", round(float((a[both] == b[both]).mean()), 3))
print(pd.crosstab(a, b).loc[cats + ["No opinion"], cats + ["No opinion"]])
weights No opinion pairs kappa within 0.84 +- 0.084 linear (i) No opinion left out 266 0.734 False linear (ii) No opinion as Neutral 429 0.620 False quadratic (i) No opinion left out 266 0.842 True quadratic (ii) No opinion as Neutral 429 0.745 False unweighted agreement on the 266 pairs: 0.669 Expert position II Completely disagree Tend to disagree Neutral Tend to agree Completely agree No opinion Expert position I Completely disagree 49 11 2 2 1 4 Tend to disagree 7 23 2 6 6 9 Neutral 3 5 2 2 4 2 Tend to agree 0 5 0 16 25 21 Completely agree 0 1 0 6 88 4 No opinion 12 9 2 21 14 65
Result: reproduced, resting on an assumption. With quadratic weights and pairs with "No opinion" left out (266 of 429 pairs), kappa is 0.842 (expected 0.84). The other readings give 0.734 (linear, left out), 0.620 (linear, as Neutral) and 0.745 (quadratic, as Neutral), all below the tolerance; the outcome rests on the quadratic weights and leaving "No opinion" out, which the paper does not state.
uni-and-landscape-placements¶
The platform placed each party on five uni-dimensions, using 18 of the 39 statements, and on a two-dimensional socio-economic and socio-cultural landscape from 0 to 100, using 16 statements; the placements (Figs. 2 and 3) conform to the anticipated placements of Slovak parties.
Assignments and polarities are those of the dataset's Supplementary Table 1. Scores are reading (a) of the criterion; reading (b) (start at 50 and move by each statement's polarity-adjusted deviation, averaged) is algebraically the same number, 50 + mean(p (x - 50)) = mean of x or 100 - x, so it is not computed apart. "No opinion" is (i) left out or (ii) scored 50. Final positions are used (Methods: "the final authority on party positioning rested solely with our Volebný Kompas team").
The expected placements below were read from the published Figures 2 and 3 of the version of record (doi:10.1038/s41597-024-03777-0) by a separate agent that saw only the figure images, not the scores computed here. It located each logo's centre in pixels and interpolated linearly between the axis ends (Fig. 2: x = 30.5 is 0 and x = 1319 is 100 in the 1350-pixel image, checked by the "You" marker at 50.0; Fig. 3: x = 51.5 to 897.5 and y = 54.5 to 901.5 for 0 to 100, top = liberal 0). In Fig. 2 the logos sit on columns at whole percentages (0, 8, 16, 25, 33, 41, 50, ...: k times 100/12 cut to an integer), read to about 0.3; the foreign policy panel has finer positions. Fig. 3 is read to about 0.3 to 1 point (PS, clipped at the top edge, and SME Rodina, partly hidden, to about 1). A party counts as matching its figure value within 1 point. Two parties are ordered differently when the figure and the scores order them opposite ways by more than 1 point each; pairs within 1 point on either side count as ties.
assign = vk.dimension_assignments(supp)
print(assign.groupby(["kind", "dimension"]).agg(statements=("statement", list), polarity=("polarity", list)).to_string())
print("statements on uni-dimensions:", assign.loc[assign.kind == "uni", "statement"].nunique(), "(expected 18)",
"| on the landscape:", assign.loc[assign.kind == "landscape", "statement"].nunique(), "(expected 16)")
# Read from the published Fig. 2 and Fig. 3 by pixel position, by an agent that had not seen the scores (see the text above).
fig2 = {
"ENVIRONMENT": {"ALIANCIA": 66, "DEMOKRATI": 33, "HLAS-SD": 41, "KDH": 33, "OLANO": 41, "PS": 83, "Republika": 8, "SME Rodina": 25, "SMER-SD": 41, "SNS": 0, "SaS": 25},
"MULTILEVEL GOVERNANCE": {"ALIANCIA": 16, "DEMOKRATI": 16, "HLAS-SD": 16, "KDH": 25, "OLANO": 41, "PS": 41, "Republika": 33, "SME Rodina": 58, "SMER-SD": 58, "SNS": 50, "SaS": 25},
"FORM OF GOVERNMENT": {"ALIANCIA": 50, "DEMOKRATI": 0, "HLAS-SD": 50, "KDH": 50, "OLANO": 50, "PS": 0, "Republika": 91, "SME Rodina": 66, "SMER-SD": 91, "SNS": 100, "SaS": 8},
"EUROPEAN UNION": {"ALIANCIA": 66, "DEMOKRATI": 83, "HLAS-SD": 83, "KDH": 41, "OLANO": 66, "PS": 100, "Republika": 8, "SME Rodina": 33, "SMER-SD": 33, "SNS": 8, "SaS": 58},
"FOREIGN POLICY": {"ALIANCIA": 70, "DEMOKRATI": 95, "HLAS-SD": 75, "KDH": 79, "OLANO": 83, "PS": 95, "Republika": 8, "SME Rodina": 75, "SMER-SD": 45, "SNS": 16, "SaS": 91},
}
fig3 = {"ECONOMIC DIMENSION": {"PS": 54.5, "SaS": 70.3, "DEMOKRATI": 52.2, "HLAS-SD": 27.1, "ALIANCIA": 38.5, "SMER-SD": 25.0, "SME Rodina": 47.6, "OLANO": 38.5, "SNS": 22.7, "Republika": 18.0, "KDH": 56.7},
"CULTURAL DIMENSION": {"PS": 0.0, "SaS": 8.2, "DEMOKRATI": 33.1, "HLAS-SD": 45.6, "ALIANCIA": 50.0, "SMER-SD": 54.1, "SME Rodina": 58.4, "OLANO": 62.3, "SNS": 74.9, "Republika": 83.0, "KDH": 74.9}}
expected = pd.DataFrame({**fig2, **fig3})
positions = pp.pivot(index="Party", columns="statement", values="Final position")
num = positions.replace(vk.LIKERT).replace("No opinion", np.nan).astype(float)
scores = {"(i) No opinion left out": vk.dimension_scores(num, assign), "(ii) No opinion as 50": vk.dimension_scores(num.fillna(50), assign)}
def discordant(x, y, tol):
# pairs of parties ordered opposite ways by the figure and the scores, each by more than the tolerance
out = []
idx = list(x.index)
for i in range(len(idx)):
for j in range(i + 1, len(idx)):
p, q = idx[i], idx[j]
dx, dy = x[p] - x[q], y[p] - y[q]
if (dx > tol and dy < -tol) or (dx < -tol and dy > tol):
out.append(f"{p}/{q}")
return out
for name, sc in scores.items():
print(f"\n{name}")
res = []
for d in expected.columns:
x, y = expected[d], sc.loc[expected.index, d]
diff = (y - x).abs()
res.append((d, round(float(diff.max()), 1), diff.idxmax(), int((diff > 1).sum()), len(discordant(x, y, 1)), discordant(x, y, 1)[:6]))
print(pd.DataFrame(res, columns=["dimension", "largest |score - figure|", "party", "parties off by more than 1", "pairs ordered differently", "examples"]).to_string(index=False))
econ = assign.dimension == "ECONOMIC DIMENSION"
print("\nEconomic axis, reading (ii), leaving out one statement of the economic dimension:")
print(pd.DataFrame({f"without statement {s}": vk.dimension_scores(num.fillna(50), assign[~(econ & (assign.statement == s))])["ECONOMIC DIMENSION"]
for s in [10, 21]}).assign(figure=expected["ECONOMIC DIMENSION"]).round(1).loc[["PS", "DEMOKRATI", "KDH", "SaS", "OLANO"]].to_string())
print("\nScores, reading (ii), with the figure values in brackets:")
s2 = scores["(ii) No opinion as 50"]
print(pd.DataFrame({d: [f"{s2.loc[p, d]:.1f} ({expected.loc[p, d]:.1f})" for p in expected.index] for d in expected.columns}, index=expected.index).to_string())
statements polarity
kind dimension
landscape CULTURAL DIMENSION [10, 11, 12, 13, 14, 15] [-1, -1, -1, 1, -1, -1]
ECONOMIC DIMENSION [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 21] [-1, -1, 1, 1, -1, -1, -1, -1, -1, -1, -1]
uni ENVIRONMENT [16, 17, 18] [1, 1, 1]
EUROPEAN UNION [25, 26, 27] [1, 1, 1]
FOREIGN POLICY [28, 29, 30, 31, 32, 33] [1, -1, 1, -1, 1, -1]
FORM OF GOVERNMENT [22, 23, 24] [1, 1, 1]
MULTILEVEL GOVERNANCE [19, 20, 21] [-1, -1, -1]
statements on uni-dimensions: 18 (expected 18) | on the landscape: 16 (expected 16)
(i) No opinion left out
dimension largest |score - figure| party parties off by more than 1 pairs ordered differently examples
ENVIRONMENT 16.0 OLANO 4 2 [DEMOKRATI/OLANO, KDH/OLANO]
MULTILEVEL GOVERNANCE 17.0 SME Rodina 7 0 []
FORM OF GOVERNMENT 0.7 Republika 0 0 []
EUROPEAN UNION 34.0 ALIANCIA 2 4 [ALIANCIA/DEMOKRATI, ALIANCIA/HLAS-SD, DEMOKRATI/OLANO, HLAS-SD/OLANO]
FOREIGN POLICY 6.0 SNS 2 0 []
ECONOMIC DIMENSION 22.7 SNS 8 3 [DEMOKRATI/PS, HLAS-SD/SMER-SD, Republika/SNS]
CULTURAL DIMENSION 12.6 SNS 4 0 []
(ii) No opinion as 50
dimension largest |score - figure| party parties off by more than 1 pairs ordered differently examples
ENVIRONMENT 0.7 ALIANCIA 0 0 []
MULTILEVEL GOVERNANCE 0.7 ALIANCIA 0 0 []
FORM OF GOVERNMENT 0.7 Republika 0 0 []
EUROPEAN UNION 0.7 ALIANCIA 0 0 []
FOREIGN POLICY 0.8 SMER-SD 0 0 []
ECONOMIC DIMENSION 6.8 PS 1 1 [DEMOKRATI/PS]
CULTURAL DIMENSION 0.3 Republika 0 0 []
Economic axis, reading (ii), leaving out one statement of the economic dimension:
without statement 10 without statement 21 figure
Party
PS 52.5 47.5 54.5
DEMOKRATI 55.0 55.0 52.2
KDH 55.0 57.5 56.7
SaS 75.0 75.0 70.3
OLANO 42.5 37.5 38.5
Scores, reading (ii), with the figure values in brackets:
ENVIRONMENT MULTILEVEL GOVERNANCE FORM OF GOVERNMENT EUROPEAN UNION FOREIGN POLICY ECONOMIC DIMENSION CULTURAL DIMENSION
ALIANCIA 66.7 (66.0) 16.7 (16.0) 50.0 (50.0) 66.7 (66.0) 70.8 (70.0) 38.6 (38.5) 50.0 (50.0)
DEMOKRATI 33.3 (33.0) 16.7 (16.0) 0.0 (0.0) 83.3 (83.0) 95.8 (95.0) 52.3 (52.2) 33.3 (33.1)
HLAS-SD 41.7 (41.0) 16.7 (16.0) 50.0 (50.0) 83.3 (83.0) 75.0 (75.0) 27.3 (27.1) 45.8 (45.6)
KDH 33.3 (33.0) 25.0 (25.0) 50.0 (50.0) 41.7 (41.0) 79.2 (79.0) 56.8 (56.7) 75.0 (74.9)
OLANO 41.7 (41.0) 41.7 (41.0) 50.0 (50.0) 66.7 (66.0) 83.3 (83.0) 38.6 (38.5) 62.5 (62.3)
PS 83.3 (83.0) 41.7 (41.0) 0.0 (0.0) 100.0 (100.0) 95.8 (95.0) 47.7 (54.5) 0.0 (0.0)
Republika 8.3 (8.0) 33.3 (33.0) 91.7 (91.0) 8.3 (8.0) 8.3 (8.0) 18.2 (18.0) 83.3 (83.0)
SME Rodina 25.0 (25.0) 58.3 (58.0) 66.7 (66.0) 33.3 (33.0) 75.0 (75.0) 47.7 (47.6) 58.3 (58.4)
SMER-SD 41.7 (41.0) 58.3 (58.0) 91.7 (91.0) 33.3 (33.0) 45.8 (45.0) 25.0 (25.0) 54.2 (54.1)
SNS 0.0 (0.0) 50.0 (50.0) 100.0 (100.0) 8.3 (8.0) 16.7 (16.0) 22.7 (22.7) 75.0 (74.9)
SaS 25.0 (25.0) 25.0 (25.0) 8.3 (8.0) 58.3 (58.0) 91.7 (91.0) 70.5 (70.3) 8.3 (8.2)
s2 = scores["(ii) No opinion as 50"]
fig, ax = plt.subplots(figsize=(5, 5))
ax.scatter(expected["ECONOMIC DIMENSION"], expected["CULTURAL DIMENSION"], marker="o", facecolors="none", edgecolors="k", label="Fig. 3, blind pixel reading")
ax.scatter(s2.loc[expected.index, "ECONOMIC DIMENSION"], s2.loc[expected.index, "CULTURAL DIMENSION"], marker="x", c="C3", label="computed (ii)")
for p in expected.index:
ax.annotate(p, (s2.loc[p, "ECONOMIC DIMENSION"], s2.loc[p, "CULTURAL DIMENSION"]), fontsize=7)
ax.axvline(50, color="0.6", lw=0.8)
ax.axhline(50, color="0.6", lw=0.8)
ax.set_xlim(0, 100); ax.set_ylim(100, 0)
ax.set_xlabel("economic left (0) to right (100)"); ax.set_ylabel("liberal (0) to conservative (100)")
ax.legend(fontsize=7)
plt.show()
print("all computed points inside 0 to 100:", bool(((s2 >= 0) & (s2 <= 100)).all().all()))
print("PS, reading (ii): economic", round(float(s2.loc["PS", "ECONOMIC DIMENSION"]), 1), "| Fig. 3", fig3["ECONOMIC DIMENSION"]["PS"],
"| PS's figure position in pixels from the centre line:", round((fig3["ECONOMIC DIMENSION"]["PS"] - 50) * (897.5 - 51.5) / 100, 1))
all computed points inside 0 to 100: True PS, reading (ii): economic 47.7 | Fig. 3 54.5 | PS's figure position in pixels from the centre line: 38.1
Result: partially reproduced. 18 and 16 statements are assigned, as stated. With "No opinion" as 50 (reading ii), every party's score on the five uni-dimensions is within 0.8 of the blind reading of Fig. 2 (the figure places them at whole percentages), and the cultural axis matches Fig. 3 for all 11 parties (largest difference 0.3), with no pair ordered differently. On the economic axis 10 parties are within 0.2 of Fig. 3, but PS scores 47.7 where the figure places it at 54.5, right of the centre line (38 pixels right of it in the published figure), so PS and DEMOKRATI (52.3; figure 52.2) are ordered differently. Leaving out statement 10 (on both axes) or 21 does not explain it: PS scores 52.5 or 47.5, still below DEMOKRATI (55.0 either way), and other parties move away from the figure. With "No opinion" left out (reading i) the placements differ from Fig. 2 on four of the five uni-dimensions. The order matches on six of the seven axes.
party-clustering¶
Hierarchical clustering of the parties' recoded positions, by Manhattan distance and average linkage, groups the parties largely as anticipated (Fig. 7).
The R code clusters recoded_party_positions.csv, the final positions with "No opinion" as 50 (README),
which is reading (a). Reading (b) leaves "No opinion" out of each pairwise distance (the distance is the mean
absolute difference over the statements both parties placed, times 39, so the scale is that of Fig. 7). The
expert-coded positions of each coder are a variant. The expected tree was read from the published Fig. 7 by
the same separate agent, blind to the computed tree: the clusters it forms, and each merge height from the
bar's pixel position against the axis ticks (1800 at y = 46.27, a linear fit of -4.180 per pixel), to about
+-3.
def clusters(Z, labels):
# every cluster of the tree as a set of labels, with its merge height
sets = [frozenset([l]) for l in labels]
out = {}
for i, j, h, _ in Z:
s = sets[int(i)] | sets[int(j)]
sets.append(s)
out[s] = h
return out
fig7 = {frozenset(c.split(",")): h for c, h in [("Republika,SNS", 474), ("SaS,DEMOKRATI", 675), ("OLANO,KDH", 725), ("PS,SaS,DEMOKRATI", 787),
("SMER-SD,Republika,SNS", 887), ("HLAS-SD,ALIANCIA", 900), ("OLANO,KDH,HLAS-SD,ALIANCIA", 1100),
("SME Rodina,OLANO,KDH,HLAS-SD,ALIANCIA", 1181), ("PS,SaS,DEMOKRATI,SME Rodina,OLANO,KDH,HLAS-SD,ALIANCIA", 1315)]}
fig7_root = 1774
def compare(Z, labels, name):
got = clusters(Z, labels)
got = {k: v for k, v in got.items() if len(k) < len(labels)}
same = [k for k in fig7 if k in got]
print(f"{name}: {len(same)} of {len(fig7)} clusters of Fig. 7 found; root height {Z[-1, 2]:.1f}")
for k in fig7:
print(f" {sorted(k)}: Fig. 7 {fig7[k]}, here {got.get(k, float('nan')):.1f}")
extra = [sorted(k) for k in got if k not in fig7]
print(" clusters here not in Fig. 7:", extra)
return len(same)
labels = [n.replace("OĽANO", "OLANO") for n in rec.index]
Za = hierarchy.linkage(pdist(rec.to_numpy(float), "cityblock"), "average")
compare(Za, labels, "(a) recoded CSV, No opinion as 50")
def nan_cityblock(M):
n = len(M)
D = np.zeros((n, n))
for i in range(n):
for j in range(i + 1, n):
ok = ~np.isnan(M[i]) & ~np.isnan(M[j])
D[i, j] = D[j, i] = np.abs(M[i, ok] - M[j, ok]).mean() * M.shape[1]
return D[np.triu_indices(n, 1)]
fin = num.loc[[l for l in labels]]
compare(hierarchy.linkage(nan_cityblock(fin.to_numpy()), "average"), labels, "(b) final positions, No opinion left out")
for coder in ["Expert position I", "Expert position II"]:
e = pp.pivot(index="Party", columns="statement", values=coder).replace(vk.LIKERT).replace("No opinion", 50).astype(float).loc[labels]
compare(hierarchy.linkage(pdist(e.to_numpy(), "cityblock"), "average"), labels, f"variant: {coder}, No opinion as 50")
fig, ax = plt.subplots(figsize=(8, 3))
hierarchy.dendrogram(Za, labels=labels, ax=ax, leaf_rotation=90)
for h in list(fig7.values()) + [fig7_root]:
ax.axhline(h, color="0.6", lw=0.6, ls="--")
ax.set_ylabel("Manhattan distance")
ax.set_title("Reading (a): recoded positions, average linkage; dashed: merge heights read from Fig. 7", fontsize=9)
plt.show()
print("root height: computed", round(float(Za[-1, 2]), 1), "| Fig. 7", fig7_root)
(a) recoded CSV, No opinion as 50: 9 of 9 clusters of Fig. 7 found; root height 1774.0 ['Republika', 'SNS']: Fig. 7 474, here 475.0 ['DEMOKRATI', 'SaS']: Fig. 7 675, here 675.0 ['KDH', 'OLANO']: Fig. 7 725, here 725.0 ['DEMOKRATI', 'PS', 'SaS']: Fig. 7 787, here 787.5 ['Republika', 'SMER-SD', 'SNS']: Fig. 7 887, here 887.5 ['ALIANCIA', 'HLAS-SD']: Fig. 7 900, here 900.0 ['ALIANCIA', 'HLAS-SD', 'KDH', 'OLANO']: Fig. 7 1100, here 1100.0 ['ALIANCIA', 'HLAS-SD', 'KDH', 'OLANO', 'SME Rodina']: Fig. 7 1181, here 1181.2 ['ALIANCIA', 'DEMOKRATI', 'HLAS-SD', 'KDH', 'OLANO', 'PS', 'SME Rodina', 'SaS']: Fig. 7 1315, here 1315.0 clusters here not in Fig. 7: [] (b) final positions, No opinion left out: 7 of 9 clusters of Fig. 7 found; root height 1886.0 ['Republika', 'SNS']: Fig. 7 474, here 471.8 ['DEMOKRATI', 'SaS']: Fig. 7 675, here 675.0 ['KDH', 'OLANO']: Fig. 7 725, here 562.5 ['DEMOKRATI', 'PS', 'SaS']: Fig. 7 787, here 724.3 ['Republika', 'SMER-SD', 'SNS']: Fig. 7 887, here 917.4 ['ALIANCIA', 'HLAS-SD']: Fig. 7 900, here 780.0 ['ALIANCIA', 'HLAS-SD', 'KDH', 'OLANO']: Fig. 7 1100, here 1132.0 ['ALIANCIA', 'HLAS-SD', 'KDH', 'OLANO', 'SME Rodina']: Fig. 7 1181, here nan ['ALIANCIA', 'DEMOKRATI', 'HLAS-SD', 'KDH', 'OLANO', 'PS', 'SME Rodina', 'SaS']: Fig. 7 1315, here nan clusters here not in Fig. 7: [['ALIANCIA', 'DEMOKRATI', 'HLAS-SD', 'KDH', 'OLANO', 'PS', 'SaS'], ['Republika', 'SME Rodina', 'SMER-SD', 'SNS']] variant: Expert position I, No opinion as 50: 2 of 9 clusters of Fig. 7 found; root height 1503.3 ['Republika', 'SNS']: Fig. 7 474, here 500.0 ['DEMOKRATI', 'SaS']: Fig. 7 675, here nan ['KDH', 'OLANO']: Fig. 7 725, here 800.0 ['DEMOKRATI', 'PS', 'SaS']: Fig. 7 787, here nan ['Republika', 'SMER-SD', 'SNS']: Fig. 7 887, here nan ['ALIANCIA', 'HLAS-SD']: Fig. 7 900, here nan ['ALIANCIA', 'HLAS-SD', 'KDH', 'OLANO']: Fig. 7 1100, here nan ['ALIANCIA', 'HLAS-SD', 'KDH', 'OLANO', 'SME Rodina']: Fig. 7 1181, here nan ['ALIANCIA', 'DEMOKRATI', 'HLAS-SD', 'KDH', 'OLANO', 'PS', 'SME Rodina', 'SaS']: Fig. 7 1315, here nan clusters here not in Fig. 7: [['ALIANCIA', 'DEMOKRATI'], ['HLAS-SD', 'SMER-SD'], ['HLAS-SD', 'SME Rodina', 'SMER-SD'], ['ALIANCIA', 'DEMOKRATI', 'KDH', 'OLANO'], ['PS', 'SaS'], ['HLAS-SD', 'Republika', 'SME Rodina', 'SMER-SD', 'SNS'], ['ALIANCIA', 'DEMOKRATI', 'KDH', 'OLANO', 'PS', 'SaS']] variant: Expert position II, No opinion as 50: 7 of 9 clusters of Fig. 7 found; root height 1568.8 ['Republika', 'SNS']: Fig. 7 474, here 600.0 ['DEMOKRATI', 'SaS']: Fig. 7 675, here nan ['KDH', 'OLANO']: Fig. 7 725, here 650.0 ['DEMOKRATI', 'PS', 'SaS']: Fig. 7 787, here 900.0 ['Republika', 'SMER-SD', 'SNS']: Fig. 7 887, here 1000.0 ['ALIANCIA', 'HLAS-SD']: Fig. 7 900, here 750.0 ['ALIANCIA', 'HLAS-SD', 'KDH', 'OLANO']: Fig. 7 1100, here nan ['ALIANCIA', 'HLAS-SD', 'KDH', 'OLANO', 'SME Rodina']: Fig. 7 1181, here 900.0 ['ALIANCIA', 'DEMOKRATI', 'HLAS-SD', 'KDH', 'OLANO', 'PS', 'SME Rodina', 'SaS']: Fig. 7 1315, here 1056.7 clusters here not in Fig. 7: [['DEMOKRATI', 'PS'], ['KDH', 'OLANO', 'SME Rodina']]
root height: computed 1774.0 | Fig. 7 1774
Result: reproduced. With the recoded positions (reading a) all 9 clusters of Fig. 7 below the root are found and no other, at heights within 3 of those read from the figure (root 1774.0, figure 1774). Leaving "No opinion" out (reading b), or using either coder's positions, changes the tree.
table2-gender, table2-age, table2-education¶
Of the responses counted in Table 2, 41.61% are female (44,919), 57.08% male (61,626) and 1.32% other or prefer not to say (1,420). / Respondents are skewed toward younger ages ... / toward the well educated ...
Readings from the criterion: (a) cleaned rows with the variable given; (b) raw rows with it given; (c) cleaned rows with all three given; (d) raw rows with all three given. Gender, age and education are empty in the same rows (data check), so (a) equals (c) and (b) equals (d). "Cleaned" is the paper's cleaning; the R code's cleaning, which also drops the 330 straight-line rows, is shown as a variant (e).
table2 = {
"GENDER": {"Female": 44919, "Male": 61626, "Other / Prefer not to say": 1420},
"AGE": {"Under 18": 1264, "18-20": 7020, "21-24": 12280, "25-34": 34894, "35-44": 27847, "45-54": 15178, "55-64": 6083, "65-74": 2716, "75 or older": 683},
"EDUCATION": {"No education": 990, "Primary education": 1665, "Secondary education": 30644, "Applied higher education": 2570,
"Vocational education": 2554, "University degree/Academic higher education": 69542},
}
table2_pct = {"Female": 41.61, "Male": 57.08, "Other / Prefer not to say": 1.32, "Under 18": 1.17, "18-20": 6.50, "21-24": 11.38, "25-34": 32.32, "35-44": 25.79,
"45-54": 14.06, "55-64": 5.63, "65-74": 2.52, "75 or older": 0.63, "No education": 0.92, "Primary education": 1.54, "Secondary education": 28.38,
"Applied higher education": 2.38, "Vocational education": 2.37, "University degree/Academic higher education": 64.41}
all3 = raw[["GENDER", "AGE", "EDUCATION"]].ne("").all(axis=1)
row_readings = {"(a) cleaned, variable given": ("paper", False), "(b) raw, variable given": ("raw", False),
"(c) cleaned, all three given": ("paper", True), "(d) raw, all three given": ("raw", True),
"(e) R code's cleaning, variable given": ("code", False)}
t2 = {}
for var, cats_ in table2.items():
out = pd.DataFrame({"paper n": pd.Series(cats_), "paper %": pd.Series({c: table2_pct[c] for c in cats_})})
for name, (s, need_all) in row_readings.items():
m = sel[s] & (all3 if need_all else raw[var].ne(""))
n = raw.loc[m, var].value_counts().reindex(list(cats_)).fillna(0).astype(int)
out[name + " n"] = n
out[name + " %"] = (100 * n / n.sum()).round(2)
t2[var] = out
print(f"\n{var}: totals", {k: int(out[k + ' n'].sum()) for k in row_readings}, "| paper total", int(out["paper n"].sum()))
print(out.to_string())
GENDER: totals {'(a) cleaned, variable given': 108063, '(b) raw, variable given': 109060, '(c) cleaned, all three given': 108063, '(d) raw, all three given': 109060, "(e) R code's cleaning, variable given": 107965} | paper total 107965
paper n paper % (a) cleaned, variable given n (a) cleaned, variable given % (b) raw, variable given n (b) raw, variable given % (c) cleaned, all three given n (c) cleaned, all three given % (d) raw, all three given n (d) raw, all three given % (e) R code's cleaning, variable given n (e) R code's cleaning, variable given %
Female 44919 41.61 44938 41.59 45358 41.59 44938 41.59 45358 41.59 44919 41.61
Male 61626 57.08 61670 57.07 62240 57.07 61670 57.07 62240 57.07 61626 57.08
Other / Prefer not to say 1420 1.32 1455 1.35 1462 1.34 1455 1.35 1462 1.34 1420 1.32
AGE: totals {'(a) cleaned, variable given': 108063, '(b) raw, variable given': 109060, '(c) cleaned, all three given': 108063, '(d) raw, all three given': 109060, "(e) R code's cleaning, variable given": 107965} | paper total 107965
paper n paper % (a) cleaned, variable given n (a) cleaned, variable given % (b) raw, variable given n (b) raw, variable given % (c) cleaned, all three given n (c) cleaned, all three given % (d) raw, all three given n (d) raw, all three given % (e) R code's cleaning, variable given n (e) R code's cleaning, variable given %
Under 18 1264 1.17 1270 1.18 1285 1.18 1270 1.18 1285 1.18 1264 1.17
18-20 7020 6.50 7033 6.51 7138 6.55 7033 6.51 7138 6.55 7020 6.50
21-24 12280 11.38 12292 11.37 12446 11.41 12292 11.37 12446 11.41 12280 11.37
25-34 34894 32.32 34914 32.31 35277 32.35 34914 32.31 35277 32.35 34894 32.32
35-44 27847 25.79 27870 25.79 28095 25.76 27870 25.79 28095 25.76 27847 25.79
45-54 15178 14.06 15182 14.05 15275 14.01 15182 14.05 15275 14.01 15178 14.06
55-64 6083 5.63 6084 5.63 6111 5.60 6084 5.63 6111 5.60 6083 5.63
65-74 2716 2.52 2719 2.52 2731 2.50 2719 2.52 2731 2.50 2716 2.52
75 or older 683 0.63 699 0.65 702 0.64 699 0.65 702 0.64 683 0.63
EDUCATION: totals {'(a) cleaned, variable given': 108063, '(b) raw, variable given': 109060, '(c) cleaned, all three given': 108063, '(d) raw, all three given': 109060, "(e) R code's cleaning, variable given": 107965} | paper total 107965
paper n paper % (a) cleaned, variable given n (a) cleaned, variable given % (b) raw, variable given n (b) raw, variable given % (c) cleaned, all three given n (c) cleaned, all three given % (d) raw, all three given n (d) raw, all three given % (e) R code's cleaning, variable given n (e) R code's cleaning, variable given %
No education 990 0.92 1015 0.94 1034 0.95 1015 0.94 1034 0.95 990 0.92
Primary education 1665 1.54 1671 1.55 1686 1.55 1671 1.55 1686 1.55 1665 1.54
Secondary education 30644 28.38 30666 28.38 30998 28.42 30666 28.38 30998 28.42 30644 28.38
Applied higher education 2570 2.38 2577 2.38 2605 2.39 2577 2.38 2605 2.39 2570 2.38
Vocational education 2554 2.37 2559 2.37 2587 2.37 2559 2.37 2587 2.37 2554 2.37
University degree/Academic higher education 69542 64.41 69575 64.38 70150 64.32 69575 64.38 70150 64.32 69542 64.41
checks = {"table2-gender": ("GENDER", list(table2["GENDER"]), list(table2["GENDER"])),
"table2-age": ("AGE", list(table2["AGE"]), ["25-34", "35-44"]),
"table2-education": ("EDUCATION", list(table2["EDUCATION"]), ["University degree/Academic higher education"])}
for claim, (var, count_cats, pct_cats) in checks.items():
out = t2[var]
for name in row_readings:
n_ok = ((out.loc[count_cats, name + " n"] - out.loc[count_cats, "paper n"]).abs() <= 0.1 * out.loc[count_cats, "paper n"] + 1e-9)
p_ok = ((out.loc[pct_cats, name + " %"] - out.loc[pct_cats, "paper %"]).abs() <= 0.1 * out.loc[pct_cats, "paper %"] + 1e-9)
largest = ((out.loc[count_cats, name + " n"] - out.loc[count_cats, "paper n"]).abs() / out.loc[count_cats, "paper n"]).max()
print(f"{claim} {name}: counts within 10% {int(n_ok.sum())}/{len(n_ok)}, percentages {int(p_ok.sum())}/{len(p_ok)}; largest relative count difference {largest:.2%}")
table2-gender (a) cleaned, variable given: counts within 10% 3/3, percentages 3/3; largest relative count difference 2.46% table2-gender (b) raw, variable given: counts within 10% 3/3, percentages 3/3; largest relative count difference 2.96% table2-gender (c) cleaned, all three given: counts within 10% 3/3, percentages 3/3; largest relative count difference 2.46% table2-gender (d) raw, all three given: counts within 10% 3/3, percentages 3/3; largest relative count difference 2.96% table2-gender (e) R code's cleaning, variable given: counts within 10% 3/3, percentages 3/3; largest relative count difference 0.00% table2-age (a) cleaned, variable given: counts within 10% 9/9, percentages 2/2; largest relative count difference 2.34% table2-age (b) raw, variable given: counts within 10% 9/9, percentages 2/2; largest relative count difference 2.78% table2-age (c) cleaned, all three given: counts within 10% 9/9, percentages 2/2; largest relative count difference 2.34% table2-age (d) raw, all three given: counts within 10% 9/9, percentages 2/2; largest relative count difference 2.78% table2-age (e) R code's cleaning, variable given: counts within 10% 9/9, percentages 2/2; largest relative count difference 0.00% table2-education (a) cleaned, variable given: counts within 10% 6/6, percentages 1/1; largest relative count difference 2.53% table2-education (b) raw, variable given: counts within 10% 6/6, percentages 1/1; largest relative count difference 4.44% table2-education (c) cleaned, all three given: counts within 10% 6/6, percentages 1/1; largest relative count difference 2.53% table2-education (d) raw, all three given: counts within 10% 6/6, percentages 1/1; largest relative count difference 4.44% table2-education (e) R code's cleaning, variable given: counts within 10% 6/6, percentages 1/1; largest relative count difference 0.00%
Result: table2-gender reproduced. Under reading (a) (cleaned rows, gender given) the counts are 44,938 female, 61,670 male and 1,455 other (41.59%, 57.07%, 1.35%), within tolerance; every reading is. The R code's cleaning gives Table 2 exactly (44,919, 61,626 and 1,420 of 107,965).
Result: table2-age reproduced. Under reading (a) every age count is within 3% of Table 2 (25-34: 34,914 against 34,894; 35-44: 27,870 against 27,847) and the two percentages match; the R code's cleaning gives every count of Table 2 exactly.
Result: table2-education reproduced. Under reading (a) every education count is within 3% of Table 2 (university 69,575, 64.38%, against 69,542, 64.41%; no education 1,015 against 990); the R code's cleaning gives every count exactly.
age-education-inconsistencies¶
The multivariate check found 40 respondents under 18 and 400 aged 18 to 20 reporting university-level education, which is not congruent with their age.
Readings: rows (a) cleaned (paper's cleaning), (b) raw, and the R code's cleaning as a variant; education (i)
"University degree/Academic higher education" only, (ii) with "Applied higher education" too. The R code's
validator rules (printed in "The deposited R code") check only that AGE and EDUCATION take one of their
categories; no rule relates the two, so the code does not contain this check.
uni = ["University degree/Academic higher education"]
rows = []
for rname, s in [("(a) cleaned", "paper"), ("(b) raw", "raw"), ("variant: R code's cleaning", "code")]:
for ename, eds in [("(i) university", uni), ("(ii) university or applied higher", uni + ["Applied higher education"])]:
m = sel[s] & raw["EDUCATION"].isin(eds)
u18 = int((m & (raw["AGE"] == "Under 18")).sum())
a1820 = int((m & (raw["AGE"] == "18-20")).sum())
rows.append((rname, ename, u18, a1820, abs(u18 - 40) <= 4 and abs(a1820 - 400) <= 40))
print(pd.DataFrame(rows, columns=["rows", "education", "under 18", "18-20", "both within 10%"]).to_string(index=False))
rows education under 18 18-20 both within 10%
(a) cleaned (i) university 27 378 False
(a) cleaned (ii) university or applied higher 34 505 False
(b) raw (i) university 27 382 False
(b) raw (ii) university or applied higher 34 509 False
variant: R code's cleaning (i) university 25 375 False
variant: R code's cleaning (ii) university or applied higher 32 500 False
Result: partially reproduced. One of the two counts is within tolerance. With the university category only, 378 respondents aged 18-20 on the cleaned rows (a, i) and 382 on the raw rows are within 40 of 400, but 27 are under 18 on both, not within 4 of 40. With applied higher education added, the counts are 34 and 505 (cleaned) and 34 and 509 (raw), so neither is within tolerance. No reading gives the under-18 count. The R code's rules do not relate age and education.
s2-s3-wide-range and s4-s5-consensus¶
S2 and S3 ... means of 43 and 42, standard deviations of 33 and medians of 25. S4 and S5 ... medians of 75 (means 70 and 69, standard deviations 26 and 29), though more than 17,435 respondents for S4 and 23,344 for S5 disagree with scores of 0 or 25.
"No opinion" is left out (the paper: "prior to the re-coding of 'no opinion' with a value of 50"); rows (a) the paper's cleaning, (b) raw rows with -1 left out of each statement; the R code's cleaning is a variant. Standard deviation with n - 1.
expect = {"S2": (43, 33, 25), "S3": (42, 33, 25), "S4": (70, 26, 75), "S5": (69, 29, 75)}
counts_paper = {"S4": 17435, "S5": 23344}
rows = []
for rname, s in [("(a) cleaned", "paper"), ("(b) raw", "raw"), ("variant: R code's cleaning", "code")]:
for st, (m, sd, med) in expect.items():
x = likert.loc[sel[s], st]
x = x[x.notna() & (x != -1)]
low = int(x.isin([0, 25]).sum())
ok = abs(x.mean() - m) <= 0.1 * m and abs(x.std(ddof=1) - sd) <= 0.1 * sd and abs(x.median() - med) <= 0.1 * med
rows.append((rname, st, len(x), round(x.mean(), 2), round(x.std(ddof=1), 2), x.median(), low, ok,
(abs(low - counts_paper[st]) <= 0.1 * counts_paper[st]) if st in counts_paper else None,
(low > counts_paper[st]) if st in counts_paper else None))
stats_t = pd.DataFrame(rows, columns=["rows", "statement", "n", "mean", "sd", "median", "n at 0 or 25", "mean, sd, median within 10%",
"(i) count within 10%", "(ii) count above paper"])
print(stats_t.to_string(index=False))
rows statement n mean sd median n at 0 or 25 mean, sd, median within 10% (i) count within 10% (ii) count above paper
(a) cleaned S2 130235 43.77 33.82 25.0 65142 True None None
(a) cleaned S3 130891 42.09 33.69 25.0 69174 True None None
(a) cleaned S4 127143 70.36 26.58 75.0 17460 True True True
(a) cleaned S5 128831 69.38 29.53 75.0 23369 True True True
(b) raw S2 131381 43.83 33.84 50.0 65601 False None None
(b) raw S3 132022 42.15 33.72 25.0 69696 True None None
(b) raw S4 128210 70.24 26.70 75.0 17804 True True True
(b) raw S5 129871 69.33 29.58 75.0 23651 True True True
variant: R code's cleaning S2 130109 43.75 33.81 25.0 65117 True None None
variant: R code's cleaning S3 130765 42.07 33.68 25.0 69149 True None None
variant: R code's cleaning S4 127017 70.36 26.56 75.0 17435 True True False
variant: R code's cleaning S5 128705 69.39 29.53 75.0 23344 True True False
Result: s2-s3-wide-range reproduced. On the cleaned rows S2 has mean 43.77, s.d. 33.82 and median 25, and S3 mean 42.09, s.d. 33.69 and median 25, all within tolerance. On the raw rows the S2 median is 50, outside it.
Result: s4-s5-consensus reproduced. On the cleaned rows S4 has mean 70.36, s.d. 26.58, median 75 and 17,460 answers of 0 or 25 (paper 17,435), and S5 mean 69.38, s.d. 29.53, median 75 and 23,369 (paper 23,344); both counts are within tolerance (reading i) and above the paper's numbers (reading ii). The R code's cleaning gives the paper's counts exactly (17,435 and 23,344), so "more than" reads as "as many as".
efa-eleven-factors¶
Parallel analyses on both the polychoric and the Pearson correlation matrices of S1 to S39 indicated that 11 factors should be extracted.
Readings of the rows, from the criterion: (a) cleaned rows, "no opinion" as 50; (b) cleaned rows, "no opinion"
missing, pairwise complete. Settings, from the criterion: psych's fa.parallel defaults (minres, 20 iterations,
the factor count). The deposited R code fixes settings the paper does not state, taken here as a variant
(author, step 1.6): rows of the R code's cleaning whose EMAIL_ID occurs once ("FILTERING DATA WITH UNIQUE
EMAIL ID"), "no opinion" as 50, fa.parallel(..., fm = "ml", fa = "fa", n.iter = 1000) for Pearson and
n.iter = 100 for polychoric. Reversing statements does not change eigenvalues, so it is not applied.
The reference eigenvalues come from normal random data with the same number of rows (for the R code's
polychoric setting, the first 100 of the 1,000 ML iterations); the generator is seeded
(a standard choice), and the reference's spread over iterations is printed to show how much the seed can move
the count.
A short synthetic check of the factor routines comes first: with a known 3-factor population correlation matrix the communalities should be recovered and parallel analysis on 2,000 simulated rows should give 3.
rng = np.random.default_rng(20240709)
Lt = np.zeros((12, 3))
Lt[np.arange(12), np.arange(12) % 3] = rng.uniform(0.5, 0.8, 12)
Lt[np.arange(12), (np.arange(12) + 1) % 3] = rng.uniform(0.0, 0.3, 12)
Rt = Lt @ Lt.T
np.fill_diagonal(Rt, 1)
for fm in ["ml", "pa"]:
Lh = vk.fa_loadings(Rt, 3, fm)
print(fm, "largest communality error:", float(np.abs((Lh ** 2).sum(1) - (Lt ** 2).sum(1)).max().round(4)))
Zs = rng.multivariate_normal(np.zeros(12), Rt, size=2000)
Rs = np.corrcoef(Zs, rowvar=False)
print("parallel analysis on simulated data:", vk.parallel_count(vk.reduced_eigenvalues(Rs, "minres"), vk.parallel_reference(2000, 12, "minres", 20, rng)), "factors (true 3)")
ml largest communality error: 0.0 pa largest communality error: 0.0011
parallel analysis on simulated data: 3 factors (true 3)
import time
n_fits_synthetic = len(vk.FITS)
rng = np.random.default_rng(39)
t0 = time.time()
mats = {}
Xa = likert.loc[sel["paper"], vk.STATEMENTS].fillna(50)
Xb = likert.loc[sel["paper"], vk.STATEMENTS]
Xc = likert.loc[sel["code_email"], vk.STATEMENTS].fillna(50)
for name, X in [("(a) cleaned, no opinion 50", Xa), ("(b) cleaned, no opinion missing", Xb), ("variant: R code's unique-email rows", Xc)]:
mats[name] = {"n": len(X), "Pearson": vk.pearson(X), "polychoric": vk.polychoric(X)}
print(f"{name}: {len(X)} rows; smallest eigenvalue Pearson {np.linalg.eigvalsh(mats[name]['Pearson']).min():.3f}, "
f"polychoric {np.linalg.eigvalsh(mats[name]['polychoric']).min():.3f}; {time.time() - t0:.0f} s")
print("largest |polychoric - Pearson|, reading (a):", float(np.abs(mats["(a) cleaned, no opinion 50"]["polychoric"] - mats["(a) cleaned, no opinion 50"]["Pearson"]).max().round(3)))
(a) cleaned, no opinion 50: 133473 rows; smallest eigenvalue Pearson 0.221, polychoric 0.109; 16 s
(b) cleaned, no opinion missing: 133473 rows; smallest eigenvalue Pearson 0.219, polychoric 0.108; 32 s
variant: R code's unique-email rows: 29493 rows; smallest eigenvalue Pearson 0.273, polychoric 0.115; 40 s largest |polychoric - Pearson|, reading (a): 0.18
refs = {}
n0 = len(vk.FITS)
for n in sorted({m["n"] for m in mats.values()}):
t0 = time.time()
sims = {}
for fm, iters in [("minres", 20), ("ml", 1000)]:
vals = []
for _ in range(iters):
Z = rng.standard_normal((n, 39), dtype=np.float32).astype(np.float64)
vals.append(vk.reduced_eigenvalues(np.corrcoef(Z, rowvar=False), fm))
sims[fm] = np.array(vals)
refs[n] = sims
print(f"n = {n}: references in {time.time() - t0:.0f} s")
ref_fits = vk.FITS[n0:]
obs_fits = []
rows = []
for name, m in mats.items():
for cor in ["Pearson", "polychoric"]:
for setting, fm, iters in [("defaults (minres, 20)", "minres", 20), ("R code (ml, %d)" % (1000 if cor == "Pearson" else 100), "ml", 1000 if cor == "Pearson" else 100)]:
n1 = len(vk.FITS)
obs = vk.reduced_eigenvalues(m[cor], fm)
obs_fits += vk.FITS[n1:]
sim = refs[m["n"]][fm][:iters]
k = vk.parallel_count(obs, sim.mean(0))
k95 = vk.parallel_count(obs, np.quantile(sim, 0.95, axis=0))
rows.append((name, cor, setting, k, k95, round(obs[k - 1], 4), round(sim.mean(0)[k - 1], 4), round(obs[k], 4), round(sim.mean(0)[k], 4)))
pa = pd.DataFrame(rows, columns=["rows", "correlation", "settings", "factors", "factors (95th percentile)", "last value above", "its reference", "next value", "its reference"])
print(pa.to_string(index=False))
print("uniqueness fits that did not converge: synthetic check", vk.FITS[:n_fits_synthetic].count(False), "of", n_fits_synthetic,
"| reference (simulated) matrices", ref_fits.count(False), "of", len(ref_fits), "| observed matrices", obs_fits.count(False), "of", len(obs_fits))
n = 29493: references in 159 s
n = 133473: references in 204 s
rows correlation settings factors factors (95th percentile) last value above its reference next value its reference
(a) cleaned, no opinion 50 Pearson defaults (minres, 20) 11 11 0.0303 0.0131 0.0042 0.0117
(a) cleaned, no opinion 50 Pearson R code (ml, 1000) 11 11 0.0344 0.0132 0.0020 0.0117
(a) cleaned, no opinion 50 polychoric defaults (minres, 20) 11 11 0.0544 0.0131 0.0103 0.0117
(a) cleaned, no opinion 50 polychoric R code (ml, 100) 11 11 0.0637 0.0132 -0.0125 0.0116
(b) cleaned, no opinion missing Pearson defaults (minres, 20) 12 12 0.0237 0.0117 -0.0315 0.0105
(b) cleaned, no opinion missing Pearson R code (ml, 1000) 11 11 0.0286 0.0132 0.0033 0.0117
(b) cleaned, no opinion missing polychoric defaults (minres, 20) 11 11 0.0422 0.0131 -0.0101 0.0117
(b) cleaned, no opinion missing polychoric R code (ml, 100) 11 11 0.0519 0.0132 -0.0135 0.0116
variant: R code's unique-email rows Pearson defaults (minres, 20) 11 11 0.0371 0.0282 0.0146 0.0250
variant: R code's unique-email rows Pearson R code (ml, 1000) 11 11 0.0382 0.0280 0.0117 0.0249
variant: R code's unique-email rows polychoric defaults (minres, 20) 11 11 0.0606 0.0282 0.0036 0.0250
variant: R code's unique-email rows polychoric R code (ml, 100) 11 11 0.0653 0.0278 0.0032 0.0249
uniqueness fits that did not converge: synthetic check 0 of 22 | reference (simulated) matrices 18 of 2040 | observed matrices 0 of 12
psych's fa.parallel with cor = "poly" on raw data, the claims file's default reading, does not compare
with simulated normal data: it resamples each column on its own (which keeps the margins and removes the
correlations) and takes polychoric correlations of the resampled data. The cell below computes that reference
for readings (a) and (b), with minres and 20 iterations, from its own seeded generator. The R code's polychoric
call passes a correlation matrix with n.obs, for which psych can only simulate normal data, as above.
rng_poly = np.random.default_rng(4139)
rows = []
for name, X in [("(a) cleaned, no opinion 50", Xa), ("(b) cleaned, no opinion missing", Xb)]:
t0 = time.time()
n0 = len(vk.FITS)
sim = vk.parallel_reference_resampled(X, "minres", 20, rng_poly)
n_ref = len(vk.FITS) - n0
bad_ref = vk.FITS[n0:].count(False)
obs = vk.reduced_eigenvalues(mats[name]["polychoric"], "minres")
k = vk.parallel_count(obs, sim.mean(0))
k95 = vk.parallel_count(obs, np.quantile(sim, 0.95, axis=0))
rows.append((name, k, k95, round(obs[k - 1], 4), round(sim.mean(0)[k - 1], 4), round(obs[k], 4), round(sim.mean(0)[k], 4),
round(sim[:, k].std(ddof=1), 4), f"{bad_ref} of {n_ref}", round(time.time() - t0)))
print(pd.DataFrame(rows, columns=["rows", "factors", "factors (95th percentile)", "last value above", "its reference", "next value",
"its reference", "sd of that reference over iterations", "reference fits not converged", "seconds"]).to_string(index=False))
rows factors factors (95th percentile) last value above its reference next value its reference sd of that reference over iterations reference fits not converged seconds
(a) cleaned, no opinion 50 11 11 0.0544 0.0159 0.0103 0.0143 0.0009 0 of 20 314
(b) cleaned, no opinion missing 11 11 0.0422 0.0168 -0.0101 0.0148 0.0012 0 of 20 304
Result: reproduced. On the cleaned rows with "no opinion" as 50 (reading a), parallel analysis with psych's defaults indicates 11 factors for both the Pearson and the polychoric matrix; the 11th Pearson factor value is above its reference (0.030 against 0.013) and the 12th below it (0.004 against 0.012). For the polychoric matrix the 12th value, 0.0106, is close to the normal-data reference (0.0117), but against psych's own reference for cor = "poly", polychoric correlations of resampled data, it is further below (0.0143, s.d. over iterations 0.0009), and the count is 11 again. With "no opinion" missing (reading b) the default counts are 12 (Pearson) and 11 (polychoric, against either reference), so the claim holds under reading (a) only with the defaults. With the R code's settings (ML, 1000 or 100 iterations) every row reading gives 11 and 11, and so do the code's unique-email rows; the 95th percentile of the reference gives the same counts as its mean. The 18 uniqueness fits that did not converge are all on simulated reference matrices (18 of 2,040); every fit on an observed matrix converged (12 of 12), and so did every fit on resampled data (40 of 40).
efa-loadings-consistent¶
With 11 factors and promax rotation, maximum likelihood and principal axis factoring gave consistent results, and S1 (real estate tax) and S3 (retirement age) loaded strongly.
From the criterion: ML and principal axis (psych fa, rotate "promax"), 11 factors, on the matrices and rows of efa-eleven-factors. S1 and S3 load strongly when their largest absolute loading is at least 0.4 (reading i) or 0.5 (reading ii) in both solutions; the solutions are consistent when, with factors matched by Tucker congruence, at least 35 of 39 statements have their largest loading on matched factors. These thresholds are the claims file's own, so an outcome rests on them.
rows = []
loadings = {}
for name, m in mats.items():
for cor in ["Pearson", "polychoric"]:
L_ml = vk.promax(vk.fa_loadings(m[cor], 11, "ml"))
L_pa = vk.promax(vk.fa_loadings(m[cor], 11, "pa"))
loadings[(name, cor)] = (L_ml, L_pa)
C = vk.tucker_congruence(L_ml, L_pa)
r, c = linear_sum_assignment(-np.abs(C))
match = dict(zip(r, c))
agree = int(sum(match[i] == j for i, j in zip(np.abs(L_ml).argmax(1), np.abs(L_pa).argmax(1))))
s1 = (np.abs(L_ml[0]).max(), np.abs(L_pa[0]).max())
s3 = (np.abs(L_ml[2]).max(), np.abs(L_pa[2]).max())
rows.append((name, cor, round(s1[0], 3), round(s1[1], 3), round(s3[0], 3), round(s3[1], 3), agree,
round(float(np.abs(C[r, c]).min()), 3), min(s1 + s3) >= 0.4 and agree >= 35, min(s1 + s3) >= 0.5 and agree >= 35))
ef = pd.DataFrame(rows, columns=["rows", "correlation", "S1 ML", "S1 PA", "S3 ML", "S3 PA", "statements on matched factors (of 39)",
"lowest congruence", "holds (i) 0.4", "holds (ii) 0.5"])
print(ef.to_string(index=False))
print("uniqueness fits that did not converge:", vk.FITS.count(False), "of", len(vk.FITS))
L_ml, L_pa = loadings[("(a) cleaned, no opinion 50", "Pearson")]
print("\nPromax loadings, reading (a), Pearson, ML (|loading| >= 0.3 shown):")
print(pd.DataFrame(np.where(np.abs(L_ml) >= 0.3, L_ml, np.nan), index=vk.STATEMENTS, columns=[f"F{i + 1}" for i in range(11)]).round(2).to_string(na_rep=""))
rows correlation S1 ML S1 PA S3 ML S3 PA statements on matched factors (of 39) lowest congruence holds (i) 0.4 holds (ii) 0.5
(a) cleaned, no opinion 50 Pearson 0.523 0.502 0.598 0.603 35 0.892 True True
(a) cleaned, no opinion 50 polychoric 0.547 0.580 0.651 0.846 32 0.355 False False
(b) cleaned, no opinion missing Pearson 0.553 0.539 0.598 0.603 35 0.901 True True
(b) cleaned, no opinion missing polychoric 0.576 0.524 0.652 0.739 32 0.358 False False
variant: R code's unique-email rows Pearson 0.451 0.379 0.643 0.656 33 0.053 False False
variant: R code's unique-email rows polychoric 0.449 0.481 0.656 0.829 36 0.235 True False
uniqueness fits that did not converge: 18 of 2122
Promax loadings, reading (a), Pearson, ML (|loading| >= 0.3 shown):
F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 F11
S1 -0.52
S2 -0.34 0.70
S3 -0.60
S4 -0.30 0.31
S5 0.68
S6 0.45
S7 -0.54
S8 -0.71
S9 -0.57
S10 -0.32
S11 -0.74
S12 -0.59
S13 0.79
S14
S15 -0.75
S16 -0.61
S17 -0.78
S18 -0.67
S19 0.32
S20 0.47
S21 -0.41
S22 0.50
S23 -0.35
S24
S25 0.82
S26
S27
S28 0.32 0.46
S29 -0.76
S30 0.83
S31
S32 0.97
S33 -0.48
S34 -0.78
S35 -0.78
S36
S37 -0.70
S38 0.69
S39 0.67
Result: partially reproduced, resting on an assumption. On the Pearson matrix of the cleaned rows (reading a), S1 loads 0.523 (ML) and 0.502 (PA) and S3 0.598 and 0.603, and 35 of 39 statements fall on matched factors (lowest congruence 0.892), so the claim holds there at both thresholds; reading (b) gives the same (S1 0.553 and 0.539, 35 of 39). On the polychoric matrices only 32 of 39 statements match (lowest congruence 0.355 and 0.358), so the two solutions are not consistent by the claims file's rule. The criterion takes the matrices of efa-eleven-factors, both Pearson and polychoric, so the claim holds on one of the two. The R code runs ML and PA only on the Pearson matrix; on its unique-email rows S1 loads 0.379 in the PA solution, with 33 of 39 matched. The outcome rests on the claims file's thresholds, which the paper does not give.
Summary¶
| Claim | Outcome |
|---|---|
| user-responses-count | reproduced |
| collection-window | partially reproduced |
| party-positions-structure | reproduced |
| eleven-parties | reproduced |
| party-self-placements | reproduced |
| coder-agreement-kappa | reproduced (assumed: quadratic weights, No opinion left out) |
| uni-and-landscape-placements | partially reproduced |
| party-clustering | reproduced |
| table2-gender, table2-age, table2-education | reproduced |
| age-education-inconsistencies | partially reproduced |
| s2-s3-wide-range, s4-s5-consensus | reproduced |
| efa-eleven-factors | reproduced |
| efa-loadings-consistent | partially reproduced (assumed: the claims file's thresholds) |
| selected-parties-top-vote | blocked |
| website-views | blocked |
A failure to reproduce here may be a failure of this attempt, not of the paper: see the reasoning and the differences from the paper's method.
How the analysis differs from the paper's: it is in Python, not R; the parallel analysis and the factor
extraction are reimplemented after psych (see vk.py), and the polychoric correlations use thresholds from each
statement's margins without psych's correction for empty cells. Figures 2, 3 and 7 were compared by values read
from the published figures by pixel position, by an agent that had not seen the computed values.
What was found about the dataset: the party workbook numbers statements 16 to 24 wrongly ("Statement no." does
not match the wording in 99 rows); the R code removes 330 straight-line responses that the paper's cleaning does
not mention, and with that step Table 2 and the S4/S5 counts are reproduced exactly; of the age-education counts of the structural validation (40 and 400) only the second is reproduced, and the R code holds no such check; 432 responses are dated after
election day, including the sample row of Table 1; "no opinion" is NA in the file, not an empty string; the
self-placement source column is empty; Fig. 3 places PS right of centre on the economic axis, where the deposited
final positions give 47.7.