Source author record

Ka Wong

Ka Wong appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

Applications Artificial Intelligence Machine Learning

Catalog footprint

What is connected

2works

3topics

4close collaborators

Actions

Connect this record

Open graph Browse works

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2022arXiv

k-Rater Reliability: The Correct Unit of Reliability for Aggregated Human Annotations

Since the inception of crowdsourcing, aggregation has been a common strategy for dealing with unreliable data. Aggregate ratings are more reliable than individual ones. However, many natural language processing (NLP) applications that rely on aggregate ratings only report the reliability of individual ratings, which is the incorrect unit of analysis. In these instances, the data reliability is under-reported, and a proposed k-rater reliability (kRR) should be used as the correct data reliability for aggregated datasets. It is a multi-rater generalization of inter-rater reliability (IRR). We conducted two replications of the WordSim-353 benchmark, and present empirical, analytical, and bootstrap-based methods for computing kRR on WordSim-353. These methods produce very similar results. We hope this discussion will nudge researchers to report kRR in addition to IRR.

preprint2015arXiv

Voronoi residual analysis of spatial point process models with applications to California earthquake forecasts

Many point process models have been proposed for describing and forecasting earthquake occurrences in seismically active zones such as California, but the problem of how best to compare and evaluate the goodness of fit of such models remains open. Existing techniques typically suffer from low power, especially when used for models with very volatile conditional intensities such as those used to describe earthquake clusters. This paper proposes a new residual analysis method for spatial or spatial-temporal point processes involving inspecting the differences between the modeled conditional intensity and the observed number of points over the Voronoi cells generated by the observations. The resulting residuals can be used to construct diagnostic methods of greater statistical power than residuals based on rectangular grids. Following an evaluation of performance using simulated data, the suggested method is used to compare the Epidemic-Type Aftershock Sequence (ETAS) model to the Hector Mine earthquake catalog. The proposed residuals indicate that the ETAS model with uniform background rate appears to slightly but systematically underpredict seismicity along the fault and to overpredict seismicity in along the periphery of the fault.