MLST 2019

Welcome to MLST 2019

The International Workshop on Machine Learning and Software Testing (MLST 2019) seeks to bring together researchers and practitioners to exchange and discuss the most recent synergistic machine learning (ML) and software testing (ST) techniques and practices. With the recent tremendous success of machine learning in many cutting-edge applications, machine learning has become a key driving force of the next-generation innovated technology. However, the quality assurance of ML software is still at a very early stage. This year’s MLST centers around two key scopes to bring researchers with diverse background (e.g., ST, AI, Security) to come up with in-depth discussion and solutions for both ML and ST: (1) How to better take the advantage for ML to further improve ST, and (2) How to better test and analyze ML towards creating ML software with better quality. As we see that machine learning has already significantly contributed to SE communities, with some initial work to ST. On the other hand, ST for ML is still at a very early stage.

MLST 2019 will, therefore, be a workshop, which seeks to develop a cross-domain community that systematically looks into both areas from the new perspective. The workshop will explore not only how we apply machine learning to ST, but also the emerging ST techniques and tools for assessing, predicting, and improving the safety, security, quality, and reliability of machine learning based software that in turn help the development of artificial intelligence. We hope MLST could facilitate to create intelligent software with high quality, as well as accelerate the process of software development and quality assurance with intelligence.

Workshop theme, goals, and relevance

Theme: The theme of the workshop is to leverage traditional software testing (ST) to better understand machine learning based software and draw strong connections between the two. We aim to apply ST to guide test case generation for machine learning software, as well as use machine learning techniques to scale up ST tasks with higher intelligence.

Goals: Our main goal is to shed light on the direction of applying the principles of software testing to machine learning and therefore evaluate the robustness of machine learning especially deep learning software. The workshop also aims to leverage machine learning techniques to advance the efficiency, accuracy, effectiveness, and usefulness of current ST techniques.

Relevance: The audience of ICST focuses on general software test generation, which is highly related and would be benefited by leveraging machine learning techniques to accelerate the process. In addition, it would be helpful for the ICST community to know how to apply current software testing generation principles to guide testing generation for machine learning software and make a broader impact to the community.


April 23, 2019
8:40 - 9:00 Opening Remarks
9:00 - 10:00 TBD
10:00 - 10:30 Coffee Break
Session 1
10:30 - 10:50 Research Paper#1 and discussion
10:50 - 11:10 Research Paper#2 and discussion
11:10 - 11:30 Research Paper#3 and discussion
11:30 - 11:50 Research Paper#4 and discussion
12:00 - 13:30 Lunch
Session 2
13:30 - 14:20 TBD
14:20 - 14:40 Research Paper#5 and discussion
14:40 - 15:00 Research Paper#6 and discussion
15:30 - 15:50 Research Paper#7 and discussion
15:50 - 16:10 Coffee Break
16:10 - 16:40 TBD


MLST workshop aims to cover the interdisciplinary topics as it relates to both ML and ST. Prospective participants are expected to focus on recent progress, breakthroughs in ML and ST, or a research vision/position statement, or industrial relevance, empirical studies as well as experience reports on either or both of the following perspectives:

  • Applying ML to ST – including but not limited to empirical studies, experience report, test requirements, test design, test automation, debugging, etc, with the ML techniques involved.
  • Applying ST to ML – including but not limited to formal verification, test design, test criteria, measurement, performance, reliability, test automation, debugging, theory of software testing, empirical studies, experience report, visions of ML software.
  • Scope and Topics

    Specific topics of interest include, but are not limited to the following subject categories:

  • Testing and verification of machine learning systems
  • Machine learning robustness, adversarial attack, defense
  • Defects, errors, failures, defects, bugs of both ML model and framework
  • Reliability, availability and safety of machine learning
  • Machine learning quality and productivity
  • Machine learning security
  • Systems (software and hardware) engineering of machine learning
  • Metrics, measurements and prediction of machine learning software quality
  • Machine learning software interpretation and understanding
  • Empirical studies using qualitative, quantitative of machine learning
  • Supporting tools and automation
  • Industry best practices
  • Machine learning for software defect prediction
  • Machine learning for test case management
  • Machine learning for debugging
  • Applications of machine learning to software verification and validation
  • This workshop accepts regular research papers within 6 pages, and short papers (new idea and position) within 4 pages.

    Submitted papers must conform to the two-column IEEE conference publication format. Templates for LaTeX and Microsoft Word are available from please use the letter format template and conference option.

    Papers should be submitted in the PDF format: they must not exceed page limit. Submissions will be handled via EasyChair. Papers must neither have been previously accepted for publication nor be under submission in another conference or journal. For your paper to be published in the proceedings, at least one of the authors of the paper must register for the conference and confirm that she/he will present the paper in person. All accepted papers will be part of the ICST joint workshop proceedings published in the IEEE Digital Library.


    All papers will be evaluated in terms of the following criteria:

  • Originality or potential for impact: The submission presents a particularly novel collation of historical work, insight or approach towards new/future work, and/or is potentially disruptive of current practice or common knowledge.
  • Soundness: The submission makes a coherent argument, substantiated by historical analysis, cogent analytical argument, or appropriately-scoped initial empirical results.
  • Relevance: The submission appropriately considers and puts itself in context with respect to the relevant literature.
  • Important Dates

  • Paper submission: January 15, 2019 January 22, 2019
  • Notification: February 5, 2019
  • Camera-ready: February 15, 2019
  • Submission Site

    Submissions will be handled via EasyChair: