BZPEERReading, trust and collaboration for research
Menu

Discover

GraphFollow connections around a paper, topic or saved view.

Workspaces

HomePick up where your research flow left off.InboxHandle requests, replies and notifications from one place.CollectionsCurate reading lists, evidence sets and shareable dossiers.ResearchSee your private provenance graph, trail and imports.CollaborationMove from discovery into focused discussion rooms.PublishDraft, preview and publish full-text research articles.

Network

ExpertsFind specialists worth following or asking for help.CommunitiesJoin research circles organized around shared work.PartnersSee external organizations active around the same topics.

Opportunities

ListingsBrowse roles, calls and collaborations with clearer fit signals.

Account

Log inReturn to your reading and collaboration workspace.Create accountStart saving work, following people and joining teams.
Log inCreate account

Researcher profile

Charles Twardy

Charles Twardy contributes to research discovery and scholarly infrastructure.

ResearcherAffiliation not importedOpen to collaborate
Artificial IntelligenceSocial and Information NetworksComputer Science and Game Theorycs.CYq-fin.TR

Trust snapshot

Quick read

Trust 17 - UnverifiedVerification L1Unclaimed author
4works
0followers
5topics
4close collaborators

Actions

Decide how to stay connected

Follow researcher0

Identity and collaboration

How to connect with this researcher

Claiming links this public author record to a researcher profile and unlocks direct collaboration workflows.

Log in to claim

Direct collaboration

Open a focused conversation when the fit is right

Claim this author entity first to unlock direct invitations.

Research graph

See the researcher in context

Open full explorer

Inspect adjacent work, topics, institutions and collaborators without jumping out to a separate graph page.

Building this graph slice

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

4 published item(s)

preprint2020arXiv

Replication Markets: Results, Lessons, Challenges and Opportunities in AI Replication

The last decade saw the emergence of systematic large-scale replication projects in the social and behavioral sciences, (Camerer et al., 2016, 2018; Ebersole et al., 2016; Klein et al., 2014, 2018; Collaboration, 2015). These projects were driven by theoretical and conceptual concerns about a high fraction of "false positives" in the scientific publications (Ioannidis, 2005) (and a high prevalence of "questionable research practices" (Simmons, Nelson, and Simonsohn, 2011). Concerns about the credibility of research findings are not unique to the behavioral and social sciences; within Computer Science, Artificial Intelligence (AI) and Machine Learning (ML) are areas of particular concern (Lucic et al., 2018; Freire, Bonnet, and Shasha, 2012; Gundersen and Kjensmo, 2018; Henderson et al., 2018). Given the pioneering role of the behavioral and social sciences in the promotion of novel methodologies to improve the credibility of research, it is a promising approach to analyze the lessons learned from this field and adjust strategies for Computer Science, AI and ML In this paper, we review approaches used in the behavioral and social sciences and in the DARPA SCORE project. We particularly focus on the role of human forecasting of replication outcomes, and how forecasting can leverage the information gained from relatively labor and resource-intensive replications. We will discuss opportunities and challenges of using these approaches to monitor and improve the credibility of research areas in Computer Science, AI, and ML.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Interval Elicitation of Forecasts in a Prediction Market Reveals Lack of Anchoring "Bias"

In an online prediction market, forecasters who could not see the current state of the market until they made their own separate estimates moved their estimates closer to the market forecast when the current state of the market became known. Their first edits to the market forecast were very similar to the first edits of forecasters who could always see the current state of the market, and forecasters in both conditions had similar accuracy. These results suggest that our more elaborate forecast elicitation method might not improve forecasts and that any anchoring on the state of the market does not constitute an error in judgment.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Trade-based Asset Model using Dynamic Junction Tree for Combinatorial Prediction Markets

Prediction markets have demonstrated their value for aggregating collective expertise. Combinatorial prediction markets allow forecasts not only on base events, but also on conditional and/or Boolean combinations of events. We describe a trade-based combinatorial prediction market asset management system, called Dynamic Asset Cluster (DAC), that improves both time and space efficiency over the method of, which maintains parallel junction trees for assets and probabilities. The basic data structure is the asset block, which compactly represents a set of trades made by a user. A user's asset model consists of a set of asset blocks representing the user's entire trade history. A junction tree is created dynamically from the asset blocks to compute a user's minimum and expected assets.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Probability and Asset Updating using Bayesian Networks for Combinatorial Prediction Markets

A market-maker-based prediction market lets forecasters aggregate information by editing a consensus probability distribution either directly or by trading securities that pay off contingent on an event of interest. Combinatorial prediction markets allow trading on any event that can be specified as a combination of a base set of events. However, explicitly representing the full joint distribution is infeasible for markets with more than a few base events. A factored representation such as a Bayesian network (BN) can achieve tractable computation for problems with many related variables. Standard BN inference algorithms, such as the junction tree algorithm, can be used to update a representation of the entire joint distribution given a change to any local conditional probability. However, in order to let traders reuse assets from prior trades while never allowing assets to become negative, a BN based prediction market also needs to update a representation of each user's assets and find the conditional state in which a user has minimum assets. Users also find it useful to see their expected assets given an edit outcome. We show how to generalize the junction tree algorithm to perform all these computations.

0 citations0 reviewsNo code linkedOpen access