BZPEERPapers, people, evidence and research pathsDiscover
Research tools
Network
Opportunities
Account
Source author record
Lifu Zhang appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.
Catalog footprint
Research graph
Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.
BZPEER is loading the nearby papers, people, topics and institutions for this page.
Published work
Leader election is a crucial process in many areas such as cloud computing, distributed systems, task orchestration, and blockchain. Oftentimes, in a distributed system, the network needs to choose a leader, which would be responsible for synchronization between different processors, data storage, information distribution, and more.
Causal decomposition depicts a cause-effect relationship that is not based on the concept of prediction, but based on the phase dependence of time series. It has been validated in both stochastic and deterministic systems and is now anticipated for its application in the complex dynamic process. Here, we present an extension of causal decomposition in the mutual complex dynamic process: cause and effect of time series are inherited in the decomposition of intrinsic components in a similar time scale. Furthermore, we illustrate comparative studies with predominate methods used in neuroscience, and show the applicability of the method particularly to physiological time series in brain-muscle interactions, implying the potential to the causality analysis in the complex physiological process.
The growth in the complexity of Convolutional Neural Networks (CNNs) is increasing interest in partitioning a network across multiple accelerators during training and pipelining the backpropagation computations over the accelerators. Existing approaches avoid or limit the use of stale weights through techniques such as micro-batching or weight stashing. These techniques either underutilize of accelerators or increase memory footprint. We explore the impact of stale weights on the statistical efficiency and performance in a pipelined backpropagation scheme that maximizes accelerator utilization and keeps memory overhead modest. We use 4 CNNs (LeNet-5, AlexNet, VGG and ResNet) and show that when pipelining is limited to early layers in a network, training with stale weights converges and results in models with comparable inference accuracies to those resulting from non-pipelined training on MNIST and CIFAR-10 datasets; a drop in accuracy of 0.4%, 4%, 0.83% and 1.45% for the 4 networks, respectively. However, when pipelining is deeper in the network, inference accuracies drop significantly. We propose combining pipelined and non-pipelined training in a hybrid scheme to address this drop. We demonstrate the implementation and performance of our pipelined backpropagation in PyTorch on 2 GPUs using ResNet, achieving speedups of up to 1.8X over a 1-GPU baseline, with a small drop in inference accuracy.