[fuzzer] Relax the FileCheck string for test fuzzer-custommutator.test (#66343)
The FileCheck string `LLVMFuzzerCustomMutatorLongSequence: {{.*}} MS: {{[0-9]*}} {{(([a-zA-Z]*-){11,})}} {{.*}}` is too restrictive and may fail the test in some case. If we look at the commit that added this check(https://github.com/llvm/llvm-project/commit/66df98945e08906ce4a057245fda81f631cfd3ae), This check is for printing out the long mutation sequence, such as this one ``` #53552 REDUCE cov: 6 ft: 6 corp: 5/9b lim: 4096 exec/s: 0 rss: 37Mb L: 2/3 MS: 54 ChangeByte-PersAutoDict-ChangeBit-ChangeBinInt-ChangeBit-ChangeBit-ChangeByte-CMP-EraseBytes-EraseBytes-CrossOver-InsertRepeatedBytes-ChangeByte-EraseBytes-InsertRepeatedBytes-ShuffleBytes-ChangeByte-ShuffleBytes-ChangeBit-CrossOver-ChangeBit-ShuffleBytes-ChangeBinInt-ShuffleBytes-EraseBytes-InsertByte-Custom-ShuffleBytes-CopyPart-InsertRepeatedBytes-PersAutoDict-InsertRepeatedBytes-ChangeByte-CrossOver-CrossOver-PersAutoDict-PersAutoDict-EraseBytes-ChangeBit-CopyPart-ChangeByte-CopyPart-InsertRepeatedBytes-CrossOver-CrossOver-CrossOver-CrossOver-ShuffleBytes-EraseBytes-InsertByte-InsertRepeatedBytes-CrossOver-EraseBytes-Custom- DE: "\377\377"-"\001\000"-"\001\000"-"\000\000\000\000\000\000\000\000"-"\001\000\000\000"- ``` But if we look at the code doing the printing ```cpp void MutationDispatcher::PrintMutationSequence(bool Verbose) { Printf("MS: %zd ", CurrentMutatorSequence.size()); size_t EntriesToPrint = Verbose ? CurrentMutatorSequence.size() : std::min(kMaxMutationsToPrint, CurrentMutatorSequence.size()); for (size_t i = 0; i < EntriesToPrint; i++) Printf("%s-", CurrentMutatorSequence[i].Name); if (!CurrentDictionaryEntrySequence.empty()) { Printf(" DE: "); EntriesToPrint = Verbose ? CurrentDictionaryEntrySequence.size() : std::min(kMaxMutationsToPrint, CurrentDictionaryEntrySequence.size()); for (size_t i = 0; i < EntriesToPrint; i++) { Printf("\""); PrintASCII(CurrentDictionaryEntrySequence[i]->GetW(), "\"-"); } } } ``` We can see that the `DE: XXX` is not always printed. So the following output is possible(and is from real-life failure), notince the missing of `DE: XXX`. ``` #13613 NEW cov: 5 ft: 5 corp: 4/6b lim: 4096 exec/s: 0 rss: 32Mb L: 2/2 MS: 27 InsertByte-ChangeBinInt-ChangeBinInt-CrossOver-ShuffleBytes-ChangeBit-EraseBytes-ShuffleBytes-InsertByte-InsertRepeatedBytes-CopyPart-InsertByte-ChangeByte-ChangeBit-InsertByte-CrossOver-EraseBytes-CopyPart-ShuffleBytes-EraseBytes-InsertByte-InsertRepeatedBytes-CrossOver-CrossOver-ShuffleBytes-ChangeBit-Custom- #13765 ...... ``` This output is totally legit and will fail that check. So I remove the check for the following strings, I think `MS: {{[0-9]*}} {{(([a-zA-Z]*-){11,})}}` is sufficient for checking the long mutation sequence. This should help resolve the flaky failure of fuzzer-custommutator.test.
Loading
Please sign in to comment