Antibiotic development and treatment are based on in vitro measures of bacterial growth inhibition, with growth indicating drug resistance. Much less is known about the determinants of antibiotic killing and the consequences of prolonged bacterial survival, also known as drug tolerance. We introduce Antimicrobial Single-Cell Testing, an imaging-based strategy for assessing antibiotic killing, and apply it to Mycobacterium abscessus, an increasingly prevalent and frequently incurable pathogen. By tracking millions of individual bacteria and analysing over 20,000 time-kill curves, we find that antibiotic tolerance is distinct from resistance but similarly associated with treatment outcomes. We show that drug tolerance is conserved across drugs with a similar mode of action and driven by the bacterial genetic background. Using a genome-wide association approach we identified molecular mechanisms which likely facilitate bacterial survival and antibiotic treatment failures. Our work establishes a scalable framework to quantify antibiotic killing, thereby offering targets for drug discovery and providing metrics for drug development and clinical decision making.