2018 International Symposium on Benchmarking, Measuring and Optimizing (Bench’ 18)


Dec 10 - Dec 13, 2018 @ Seattle, WA, USA

Colocated with IEEE Big Data 2018


Homepage

Benchmarking, measuring, and optimizing are fundamental activities in human being's lives. Starting as nine BPOE Workshop series and SDBA Workshop in conjunction with ASPLOS, VLDB, and ICS, this symposium (Bench 18) is dedicated to benchmarking, measuring, and optimizing complex systems, including (but not limited to) Big Data, AI, chainblock, datacenter, cloud and warehouse-scale computing, high performance computing, Mobile Robotics, edge and fog computing,Big Scientific data, IoT, and the other miscellaneous things, i.e., education systems, financial systems, and power systems. In addition, we encourage publishing benchmark specifications and the summaries of the open-source benchmarking projects.

Program

Dec 10 Morning Keynote speak sessions
Two keynote speakers
Dec 10 Morning BenchCouncil Open Meeting
Two invited talks
Dec 10 Afternoon Technical paper sessions
Six presentations
Dec 11 Morning Keynote speak sessions
Two keynote speakers
Dec 11 Morning BenchCouncil Open Meeting
Two invited talks
Dec 11 Afternoon Technical paper sessions
Six presentations
Dec 12 Morning BenchCouncil Open Meeting
Two invited talks
Dec 12 Morning Breakout sessions
Focus on each benchmarking proposal
Dec 12 Afternoon Breakout sessions
Focus on each benchmarking proposal
Dec 12 Afternoon Breakout reports
Report the discussion results and plan of each benchmarking proposal
Dec 13 Morning Tutorial
BigDataBench Tutorial: a scalable and unified Big Data and AI benchmark suite
Dec 13 Morning Closing the symposium

Important Dates


Submission Deadline (Submission Site)

Time Event
Sep 20, 2018 Oct 10, 2018 Electronic submission of full papers
Nov 9, 2018 Notification of paper acceptance
Dec 10-12, 2018 Conference
Feb 9, 2019 Camera-ready of extended papers

Call for Benchmarking Proposals

Benchmarking proposals are solicited for BenchCouncil open meeting, which will take place in Seattle, WA, on Dec 10-13. Benchmark proposals are solicited on any topic that is relevant to Bench 2018 audience, including (but not limited to):

  • Artificial Intelligence

  • NewSQL and distributed database systems

  • Blockchain

  • Datacenter, cloud and warehouse-scale computing

  • High performance computing

  • Mobile Robotics

  • Egde and fog computing

  • Big Scientific Data

  • Internet of Things (IoT)

  • Education, financial and power system

  • Once a proposal is accepted, at least one of the authors should present their proposal. Meanwhile, a breakout session will be organized to discuss the benchmarking targets, major challenges, and plans. Finally, for each invited talk, a breakout report will be presented by the same speaker.
    After the conference, the authors should form a position paper that will be published in the proceeding of Bench 2018. Finally, for each benchmarking plan, a working group will be established, and each published position paper will act as a starting point for a formal benchmark specification, that will be released by BenchCouncil in future.


    Proposals should provide the following information:

    • Title.

    • Authors and contact info.

    • 1 paragraph abstract suitable for benchmark publicity.

    • biography per author suitable for publicity.

    Proposals should be submitted in PDF format via e-mail to Dr. Chen Zheng (zhengchen@ict.ac.cn) and Dr. Zhen Jia (zhenj@cs.princeton.edu) with the subject "BenchCouncil Proposal". Submissions will be acknowledged via e-mail.


    Important Dates

    Submission deadline: Nov 10, 2018
    Notification: Nov 13, 2018

Call for Papers

Topics of Interest

We solicit papers in all areas related to benchmarking, measuring, and optimizing systems. Short papers are encouraged, and the authors can extend their short papers to full versions after the conference. Topics of interest include (but are not limited to):

  • Big data domains

  • Artificial intelligence

  • NewSQL and distributed database systems

  • Blockchain

  • Datacenter, cloud and warehouse-scale computing

  • High performance computing

  • Mobile Robotics

  • Edge and fog computing

  • Big scientific data

  • Internet of Things (IoT)

  • Education, financial and power system

Submission Site

Bench 2018 Submission Site:
https://wi-lab.com/cyberchair/2018/bigdata18/scripts/submit.php?subarea=SP04&undisplay_detail=1&wh=/cyberchair/2018/bigdata18/scripts/ws_submit.php

Papers must be submitted in PDF. Short papers (no more than 4 pages in standard two-column IEEE conference format or 6 pages in LNCS format, not including references) are encouraged, and the authors can extend their short paper to 12 pages in LNCS format, not including references, in the final publication. If a full paper is submitted, the page limit is 8 pages in standard two-column IEEE conference format or 12 pages in LNCS format, not including references. The submissions will be judged based on the merit of the ideas rather than the length. After the conference, revised papers will be published by Springer LNCS (www.springer.com/lncs, indexed by EI). Please note that LNCS format is the final one for publishing.


Formatting Instructions
IEEE conference format:
8.5" x 11" (DOC, PDF)
LaTex Formatting Macros

LNCS format:
LaTex Template

Please make sure your submission satisfies ALL of the following requirements:

  • Paper must be submitted in printable PDF format.

  • Please number the pages of your submission.

  • The submission must be formatted for black-and-white printers. Please make sure your figures are readable when printed in black-and-white.

  • The submission must describe unpublished work that is not currently under review of any other conference or journal venues.

Call for Tutorial

Tutorial proposals are solicited for Bench 2018, which will take place in Seattle, WA. Tutorials will be held on Dec 10-12.
Proposals for both half- and full-day tutorials are solicited on any topic that is relevant to the Bench 2018 audience. In our experiences, tutorials seeking to achieve either of the following goals have been particularly successful:

  • Describe an important piece of research infrastructure.

  • Educate the community on an emerging topic.


  • Submission Procedures

    Proposals should provide the following information:

    • Title.

    • Presenter(s) and contact information.

    • Proposed duration (full day, half day).

    • 1-2 paragraph abstract suitable for tutorial publicity.

    • 1 paragraph biography per presenter suitable for tutorial publicity.

    • 1-3 page description (for evaluation). This should include:

    1. Tutorial scope and objectives,
    2. Topics to be covered,
    3. Target audience,
    4. If the tutorial has been held previously, the location (i.e., conference), date, and number of attendees.

    Proposals should be submitted in PDF format via e-mail to Lei Wang (wangle_2011@ict.ac.cn ) and Chen Zheng (zhengchen@ict.ac.cn) with the subject "Bench’18 Tutorial Proposal". Submissions will be acknowledged via e-mail.


    Important Dates

    Submission deadline: Oct 6, 2018
    Notification: October 24th, 2018

Organization

Program Chair Prof. Jianfeng Zhan
ICT, Chinese Academy of Sciences and University of Chinese Academy of Sciences
Program Committee Lizy K. John, The University of Texas at Austin USA
Vijay Janapa Reddi, University of Texas at Austin USA and Google
Wenguang Chen, Tsinghua University China
Woongki Baek, UNIST Korea
Yunji Chen, ICT, CAS China
Piotr Luszczek, University of Tennessee USA
Rui Hou, Institute of Information Engineering, CAS China
Yueguo Chen, Renmin University of China
H. Peter Hofstee, IBM USA
Cheqing Jin, East China Normal University China
Trevor E. Carlson, National University of Singapore (NUS) Singapore Junzhao Du, Xidian University China
Hyogi Sim, Oak Ridge National Laboratory USA
Weining Qian, East China Normal University China
Zheng Cao, Alibaba China
Matthias Nicola, IBM Germany
Zhihui Du, Tsinghua University China
Chunjie Luo, ICT, CAS
Bin Ren, College of William and Mary USA
Hua Chen, ICT, CAS China
Jiaquan Gao, Nanjing Normal University China
Jiannan Ouyang, Facebook USA
Xueming Si, Fudan University China
Yanjun Wu, Institute of Software Chinese Academy of Sciences China
Lucas Mello Schnorr, Federal University of Rio Grande do Sul (UFRGS) Brazil
Yunquan Zhang, ICT, CAS China
Shengzhong Feng, Shenzhen Institutes of Advanced Technology, CAS China
Zhen Jia, Princeton University USA
Zhibin Yu, Shenzhen Institutes of Advanced Technology, CAS China
Dapeng Wang, Institute of Software Chinese Academy of Sciences China
Bo Wu, Colorado School of Mines USA
Shaoliang Peng, National University of Defense Technology China
Yong Qi, Xi'an jiaotong University China
Gwangsun Kim, Arm Inc., Austin, Texas USA
Rui Ren, ICT, CAS China
Zujie Ren, Hangzhou Dianzi University China
Xiaoyi Lu, The Ohio State University USA
Lei Wang, ICT, CAS China
Jianhui Li, Computer Network Information Center, CAS China
Jianwu Wang, University of Maryland, Baltimore County USA
Kenli Li, Hunan University China
Suzanne Rivoire, Sonoma State University USA Rui Han, ICT, CAS China
Jungang Xu, Chinese academy of sciences China
Xu Liu, College of William and Mary USA
Tong Wu, National Institute of Metrology, China
Li ZHA, University of the Chinese Academy of Sciences
Lei Liu, ICT, CAS China
Chen Zheng, ICT, CAS China
Tutorial Co-Chairs Dr. Lei Wang
ICT, Chinese Academy of Sciences and University of Chinese Academy of Sciences
Dr. Chen Zheng
ICT, Chinese Academy of Sciences and University of Chinese Academy of Sciences
Publication Chair Prof. Xiaohui Hu
Science & Technology on Integrated Information System Laboratory, Institute of Software Chinese Academy of Sciences
Web Chairs Mr. Fei Tang
University of Chinese Academy of Sciences
Publicity Chairs Miss. Wanling Gao, University of Chinese Academy of Sciences
Miss. Rui Ren, ICT, Chinese Academy of Sciences
Dr. Zhen Jia, Princeton University
Submission Co-Chairs Miss. Rui Wang
Science & Technology on Integrated Information System Laboratory, Institute of Software Chinese Academy of Sciences
Miss. Ruiying Li
Science & Technology on Integrated Information System Laboratory, Institute of Software Chinese Academy of Sciences
Sponsorship Chair Dr. Biwei Xie
ICT, Chinese Academy of Sciences

Previous Events

Workshop Dates Location
BPOE-1 October 7, 2013 IEEE Big Data Conference, San Jose, CA
BPOE-2 October 31, 2013 CCF HPC China, Guilin, China
BPOE-3 December 5,2013 CCF Big Data Technology Conference 2013, BeiJing, China
BPOE-4 March 1, 2014 ASPLOS 2014, Salt Lake City, Utah, USA
BPOE-5 September 5, 2014 VLDB 2014, Hangzhou, Zhejiang Province, China
BPOE-6 Spetember 4th, 2015 VLDB 2015, Hilton, Waikoloa hotel·Kohala Coast, Hawai’i
BPOE-7 April 3, 2016 ASPLOS 2016, Atlanta, Georgia, USA
BPOE-8 April 9, 2017 ASPLOS 2017, Xi’an, China
BPOE-9 March 25, 2018 ASPLOS 2018, Williamsburg, VA, USA
SDBA-1 June 12, 2018 ICS 2018, Beijing, China


1) J. Zhan, R. Han, C. Weng. Big Data Benchmarks, Performance Optimization, and Emerging Hardware, Springer LNCS, volume 8807, 2014.
2) J. Zhan, R. Han, R. V. Zicari. Big Data Benchmarks, Performance Optimization, and Emerging Hardware, Springer LNCS, volume 9495, 2016.

Location

Seattle, WA