Abstract
Many automated system analysis techniques (e.g., model checking, model-based testing, runtime monitoring) rely on ?rst obtaining a model of the system under analysis. System modeling is often done manually, which is highly non-trivial for real-world complex systems like cyber-physical systems. This is considered as a major hindrance to adopt model-based system analysis and development techniques. To overcome this problem, researchers have proposed to automat-ically “learn” models based on sample system executions. However, there are many research questions which must be solved in order to apply model learn-ing to real-world problems. For instance, does model learning pay off by pro-viding better analysis results than methods like statistical model checking; do existing model learning techniques scale to actual systems; can we provide a certain guarantee on the correctness of the analysis results based on the learned model? To answer these research questions, the thesis ?rst studies state-of-the-art model learning algorithms for model checking and conducts an empirical study based on a set of benchmark systems (including a set of randomly gen-erated models) as well as a real-world water treatment system. We identify the following issues with existing model learning techniques. First, existing learning algorithms may not generalize well. We thus propose a novel learning approach based on genetic algorithms to better control the degree of general-ization. Second, real-typed variables in real-world systems may render exist-ing learning algorithms infeasible due to the in?nite alphabet. To address this problem, we recognize the necessity of learning with abstraction and propose a counterexample-guided abstraction re?nement stylish learning framework which automatically identi?es a level of abstraction to verify/falsify a given safety property. Third, rare event is a major threat to the accuracy of learning for probability estimation. To tackle the challenge, we propose a smart sam-pling algorithm which would guide sampling towards those more interesting events and thus lead to more accurate probability estimation. Lastly, we apply the proposed techniques to verify a real-world water treatment system. The results show the validity and effectiveness of the proposed techniques. Besides the series of work on verifying the water treatment system, there are some other applications related to the proposed techniques as well. For in-stance, importance sampling is an effective rare event simulation technique but requires the full knowledge of the system. However, a model obtained by learn-ing is only an approximate model with uncertainty. We thus study importance sampling on such learned models. In addition, we also apply model learning to improve existing concolic testing techniques so as to generate relevant test cases more ef?ciently.