Caroline Lemieux

Caroline Lemieux

Post Doc Researcher
Microsoft Research NYC

Assistant Professor (Starting Fall 2022)
University of British Columbia
Department of Computer Science

Email: clemieuxbots-gotta-get-smarter-than-this@its-really-not-that-hardmicrosoft.com or clemieuxbots-gotta-get-smarter-than-this@its-really-not-that-hardcs.ubc.ca for UBC-related inquiries.
Twitter: @cestlemieux
Github: carolemieux
Oct 2021: I am recruiting MSc and PhD students for the 2022/23 Academic Year. The first step for interested students is to apply to UBC CS. Some possible research directions include: fuzzing and program synthesis to characterize bugs and exploit patterns; inference of input structures (generators, grammars) for structured fuzzing; fundamental research on the factors leading to the effectiveness of coverage-guided fuzzing. However, I am always happy to advise students on research directions of particular interest to them if they intersect with my broader research interests (improving the correctness, security, and performance of software systems).

Feel free to email me about the positions, but note I am most likely to reply to inquiries that highlight the compatibility of my research interests with your own goals.

May 2021: I will be joining UBC and the Software Practices Lab as an Assistant Professor in the Department of Computer Science in Fall 2022!

About

I am a postdoctoral researcher at Microsoft Research, NYC. My research aims to help developers improve the correctness, security, and performance of software systems. I am particularly interested in developing methods that are applicable to large, existing software systems, ranging from complex open-source projects to industrial-scale software. This leads to a broad array of research interests over software engineering and programming languages topics: test-input generation, specification mining, program synthesis. My interest in analyzing large software systems with fuzz testing touches the systems and security communities.

In May 2021, I received my Ph.D. at the University of California, Berkeley, where I was advised by Koushik Sen. My PhD research focused on automatic test-input generation, particularly fuzz testing, with the goal of producing inputs that induce correctness, security, and performance bugs in software. I also explored how the search techniques used to scale these input generation tools apply in the domain of program synthesis. My dissertation is available on the EECS department website.

In Summer 2018, I interned at Google, where I built large-scale static analysis to automatically generate fuzz targets. My team continued building on my work after my departure, resulting in the FUDGE tool. In Summer 2017 I was a research intern in the Tools for Software Engineers group at Microsoft, working on automating detection of anomolous errors in the distributed build system CloudBuild. In May 2016 I received my B.Sc. (in Combined Honours Computer Science and Mathematics) at the University of British Columbia, and was awarded the Governor General's Silver Medal for highest standing in the graduating class of the Faculty of Science.

If you are on the academic job market, you may be interested in perusing my research, teaching, and diversity statements.

Projects


Arvada Learning higher-level representations of a set of examples has applications to fuzzing, comprehension, and testing tasks, and is of interest at a fundamental language understanding level. Arvada takes in an example set and an oracle, and learns a grammar that expands that example set as much as possible. Arvada learns that grammar by trying to learn "parse trees" via a sequence of "bubble" and "merge" operations over the example set. The bubbles add structure, and the merges add generalization. Check out the Arvada code base!
Gauss Input-output examples as specifications for program synthesis are compelling as a simple communication language and for their flexibility. However, especially for table transformations, they can be tedious to fully provide, and also omit some easily-available user intent. Gauss combines a UI that creates a graph of user intent while a user builds a partial input-output example with a novel form of inductive reasoning over graphs. This allows for faster, more precise synthesis of table transformations, and opens the avenue of graphs as synthesis specification. You can try out Gauss's UI by launching the binder in the code base!
RLCheck Effective property-based testing relies on the rapid generation of many diverse valid inputs for the program under test. However, when validity constraints on inputs are complex, this requires building specialized generators for each program under test. However, if an existing generator generates a superset of all valid inputs, we should be able to guide it to generate only valid inputs. RLCheck uses a Q-table based Reinforcement Learning approach to guide generators to produce many diverse valid inputs, given a validity function. RLCheck's Java implementation is available as open-source.
AutoPandas Modern Python APIs are complex and very difficult to learn. Novice users could ask their API questions on StackOverflow, but the answer might be slow to arrive or unpersonalized... enter AutoPandas. AutoPandas is a programming-by-example synthesis engine for the Python API pandas, in particular for its dataframe transformations. To handle the complex space of API programs, AutoPandas uses a novel neural-backed generator approach to synthesizing programs in the pandas API. You can try out AutoPandas live here.
FuzzFactory FuzzFactory is an extension of AFL that generalizes coverage-guided fuzzing to domain-specific testing goals. FuzzFactory allows users to guide the fuzzer's search process without having to modify the core search algorithm. FuzzFactory's key abstraction is that of waypoints: intermediate inputs that are saved during the fuzzing loop. For example, PerfFuzz saves inputs that increase loop execution counts, a magic-byte fuzzer may save inputs that have partially correct magic bytes, or a directed fuzzer may save inputs that are more likely to exercise a program point of interest.
Zest Binary-level mutational fuzzing excels at exercising the syntactic (parsing) phase of programs, but produces few valid inputs that exercise deeper stages of the program. QuickCheck-style random testing allows us to test programs with random generators of highly-structured inputs, but does not use program feedback to bias its input generation. Zest, the default front-end of our JQF platform, leverages (1) QuickCheck-style generators to generate only syntactically valid inputs, and (2) program coverage and validity feedback to generate inputs which explore deep parts of the program.
PerfFuzz Performance problems in software can arise unexpectedly when programs are provided with inputs that exhibit pathological behavior. But how can we find these inputs in the first place? Given a program and at least one seed input, PerfFuzz automatically generates inputs that exercise pathological behavior across program locations, without any domain knowledge.
FairFuzz FairFuzz is a fuzzer built on top of AFL which targets rare branches to achieve faster program coverage. FairFuzz achieves this by (1) selectively mutating inputs which exercise branches hit by few fuzz-tester generated inputs and (2) using a mutation mask to restrict mutations of these inputs to the parts which can be mutated while still hitting the branch of interest. On our benchmarks, FairFuzz achieves program coverage than AFL or AFLFast, and has a particular advantage on programs with highly nested structure.

Past

Texada I am the main developer of the Texada tool, which mines linear temporal logic (LTL) relationships of arbitrary length and complexity from textual logs. Texada takes as input a log of traces and a property type expressed in LTL and outputs instantiations of this property types with log events which hold on the entire log. Texada also supports confidence and support thresholds to allow for mining on imperfect or incomplete logs.
Quarry I also built the Quarry tool. Quarry interfaces data predicates with temporal invariants in order to extract data-temporal invariants of arbitrary length and complexity from program execution. Quarry mines relationships between Daikon-style data predicates specified in linear temporal logic (LTL). Quarry uses Daikon for data predicate inference and Texada for inference of temporal invariants.

Publications

Research Team

I've been lucky to work with some excellent collaborators throughout my career.

Undergraduate students mentored at Berkeley:

Invited Talks

Service

Funding

I am thankful to have received funding from Google, NSERC, NSF, UBC, and UCB to support my research.