Source author record

Todor Ivanov

Todor Ivanov appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

Databases Distributed, Parallel, and Cluster Computing hep-ex physics.comp-ph

Catalog footprint

What is connected

6works

4topics

4close collaborators

Actions

Connect this record

Open graph Browse works

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2018arXiv

A Roadmap for HEP Software and Computing R&D for the 2020s

Particle physics has an ambitious and broad experimental programme for the coming decades. This programme requires large investments in detector hardware, either to build new facilities and experiments, or to upgrade existing ones. Similarly, it requires commensurate investment in the R&D of software to acquire, manage, process, and analyse the shear amounts of data to be recorded. In planning for the HL-LHC in particular, it is critical that all of the collaborating stakeholders agree on the software goals and priorities, and that the efforts complement each other. In this spirit, this white paper describes the R&D activities required to prepare for this software upgrade.

preprint2016arXiv

Evaluating Hive and Spark SQL with BigBench

The objective of this work was to utilize BigBench [1] as a Big Data benchmark and evaluate and compare two processing engines: MapReduce [2] and Spark [3]. MapReduce is the established engine for processing data on Hadoop. Spark is a popular alternative engine that promises faster processing times than the established MapReduce engine. BigBench was chosen for this comparison because it is the first end-to-end analytics Big Data benchmark and it is currently under public review as TPCx-BB [4]. One of our goals was to evaluate the benchmark by performing various scalability tests and validate that it is able to stress test the processing engines. First, we analyzed the steps necessary to execute the available MapReduce implementation of BigBench [1] on Spark. Then, all the 30 BigBench queries were executed on MapReduce/Hive with different scale factors in order to see how the performance changes with the increase of the data size. Next, the group of HiveQL queries were executed on Spark SQL and compared with their respective Hive runtimes. This report gives a detailed overview on how to setup an experimental Hadoop cluster and execute BigBench on both Hive and Spark SQL. It provides the absolute times for all experiments preformed for different scale factors as well as query results which can be used to validate correct benchmark execution. Additionally, multiple issues and workarounds were encountered and solved during our work. An evaluation of the resource utilization (CPU, memory, disk and network usage) of a subset of representative BigBench queries is presented to illustrate the behavior of the different query groups on both processing engines. Last but not least it is important to mention that larger parts of this report are taken from the master thesis of Max-Georg Beer, entitled "Evaluation of BigBench on Apache Spark Compared to MapReduce" [5].

preprint2015arXiv

Evaluating Hadoop Clusters with TPCx-HS

The growing complexity and variety of Big Data platforms makes it both difficult and time consuming for all system users to properly setup and operate the systems. Another challenge is to compare the platforms in order to choose the most appropriate one for a particular application. All these factors motivate the need for a standardized Big Data benchmark that can help the users in the process of platform evaluation. Just recently TPCx-HS [1][2] has been released as the first standardized Big Data benchmark designed to stress test a Hadoop cluster. The goal of this study is to evaluate and compare how the network setup influences the performance of a Hadoop cluster. In particular, experiments were performed using shared and dedicated 1Gbit networks utilized by the same Cloudera Hadoop Distribution (CDH) cluster setup. The TPCx-HS benchmark, which is very network intensive, was used to stress test and compare both cluster setups. All the presented results are obtained by using the officially available version [1] of the benchmark, but they are not comparable with the officially reported results and are meant as an experimental evaluation, not audited by any external organization. As expected the dedicated 1Gbit network setup performed much faster than the shared 1Gbit setup. However, what was surprising is the negligible price difference between both cluster setups, which pays off with a multifold performance return.

preprint2014arXiv

Benchmarking DataStax Enterprise/Cassandra with HiBench

This report evaluates the new analytical capabilities of DataStax Enterprise (DSE) [1] through the use of standard Hadoop workloads. In particular, we run experiments with CPU and I/O bound micro-benchmarks as well as OLAP-style analytical query workloads. The performed tests should show that DSE is capable of successfully executing Hadoop applications without the need to adapt them for the underlying Cassandra distributed storage system [2]. Due to the Cassandra File System (CFS) [3], which supports the Hadoop Distributed File System API, Hadoop stack applications should seamlessly run in DSE. The report is structured as follows: Section 2 provides a brief description of the technologies involved in our study. An overview of our used hardware and software components of the experimental environment is given in Section 3. Our benchmark methodology is defined in Section 4. The performed experiments together with the evaluation of the results are presented in Section 5. Finally, Section 6 concludes with lessons learned.

preprint2014arXiv

Performance Evaluation of Virtualized Hadoop Clusters

In this report we investigate the performance of Hadoop clusters, deployed with separated storage and compute layers, on top of a hypervisor managing a single physical host. We have analyzed and evaluated the different Hadoop cluster configurations by running CPU bound and I/O bound workloads. The report is structured as follows: Section 2 provides a brief description of the technologies involved in our study. An overview of the experimental platform, setup test and configurations are presented in Section 3. Our benchmark methodology is defined in Section 4. The performed experiments together with the evaluation of the results are presented in Section 5. Finally, Section 6 concludes with lessons learned.

preprint2013arXiv

On the inequality of the 3V's of Big Data Architectural Paradigms: A case for heterogeneity

The well-known 3V architectural paradigm for Big Data introduced by Laney (2011), provides a simplified framework for defining the architecture of a big data platform to be deployed in various scenarios tackling processing of massive datasets. While additional components such as Variability and Veracity have been discussed as an extension to the 3V model, the basic components (volume, variety, velocity) provide a quantitative framework while variability and veracity target a more qualitative approach. In this paper we argue why the basic 3V's are not equal due to the different requirements that need to be covered in case higher demands for a particular "V". Similar to other conjectures such as the CAP theorem 3V based architectures differ on their implementation. We call this paradigm heterogeneity and we provide a taxonomy of the existing tools (as of 2013) covering the Hadoop ecosystem from the perspective of heterogeneity. This paper contributes on the understanding of the Hadoop ecosystem from the perspective of different workloads and aims to help researchers and practitioners on the design of scalable platforms targeting different operational needs.