T H O M A S H. C O R M E N C H A R L E S E. L E I S E R S O N R O N A L D L. R I V E S T C L I F F O - pdf free download

40K downloads 5K Views 5MB Size

No documents

T H O M A S H. C O R M E N C H A R L E S E. L E I S E R S O N R O N A L D L. R I V E S T C L I F F O R D STEIN

INTRODUCTION TO

ALGORITHMS T H I R D

E D I T I O N

Introduction to Algorithms Third Edition

Thomas H. Cormen Charles E. Leiserson Ronald L. Rivest Clifford Stein

Introduction to Algorithms Third Edition

The MIT Press Cambridge, Massachusetts

London, England

c 2009 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form or by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher. For information about special quantity discounts, please email special [email protected] This book was set in Times Roman and Mathtime Pro 2 by the authors. Printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Introduction to algorithms / Thomas H. Cormen . . . [et al.].—3rd ed. p. cm. Includes bibliographical references and index. ISBN 978-0-262-03384-8 (hardcover : alk. paper)—ISBN 978-0-262-53305-8 (pbk. : alk. paper) 1. Computer programming. 2. Computer algorithms. I. Cormen, Thomas H. QA76.6.I5858 2009 005.1—dc22 2009008593 10 9 8 7 6 5 4 3 2

Contents

Preface

xiii

I Foundations Introduction

3

1

The Role of Algorithms in Computing 5 1.1 Algorithms 5 1.2 Algorithms as a technology 11

2

Getting Started 16 2.1 Insertion sort 16 2.2 Analyzing algorithms 23 2.3 Designing algorithms 29

3

Growth of Functions 43 3.1 Asymptotic notation 43 3.2 Standard notations and common functions

4

? 5

?

53

Divide-and-Conquer 65 4.1 The maximum-subarray problem 68 4.2 Strassen’s algorithm for matrix multiplication 75 4.3 The substitution method for solving recurrences 83 4.4 The recursion-tree method for solving recurrences 88 4.5 The master method for solving recurrences 93 4.6 Proof of the master theorem 97 Probabilistic Analysis and Randomized Algorithms 114 5.1 The hiring problem 114 5.2 Indicator random variables 118 5.3 Randomized algorithms 122 5.4 Probabilistic analysis and further uses of indicator random variables 130

vi

Contents

II Sorting and Order Statistics Introduction 6

7

8

9

147

Heapsort 151 6.1 Heaps 151 6.2 Maintaining the heap property 6.3 Building a heap 156 6.4 The heapsort algorithm 159 6.5 Priority queues 162

154

Quicksort 170 7.1 Description of quicksort 170 7.2 Performance of quicksort 174 7.3 A randomized version of quicksort 7.4 Analysis of quicksort 180 Sorting in Linear Time 191 8.1 Lower bounds for sorting 8.2 Counting sort 194 8.3 Radix sort 197 8.4 Bucket sort 200

179

191

Medians and Order Statistics 213 9.1 Minimum and maximum 214 9.2 Selection in expected linear time 215 9.3 Selection in worst-case linear time 220

III Data Structures Introduction 10

11

?

229

Elementary Data Structures 232 10.1 Stacks and queues 232 10.2 Linked lists 236 10.3 Implementing pointers and objects 10.4 Representing rooted trees 246 Hash Tables 253 11.1 Direct-address tables 254 11.2 Hash tables 256 11.3 Hash functions 262 11.4 Open addressing 269 11.5 Perfect hashing 277

241

Contents

12

? 13

14

vii

Binary Search Trees 286 12.1 What is a binary search tree? 286 12.2 Querying a binary search tree 289 12.3 Insertion and deletion 294 12.4 Randomly built binary search trees 299 Red-Black Trees 308 13.1 Properties of red-black trees 13.2 Rotations 312 13.3 Insertion 315 13.4 Deletion 323

308

Augmenting Data Structures 339 14.1 Dynamic order statistics 339 14.2 How to augment a data structure 14.3 Interval trees 348

345

IV Advanced Design and Analysis Techniques Introduction

357

15

Dynamic Programming 359 15.1 Rod cutting 360 15.2 Matrix-chain multiplication 370 15.3 Elements of dynamic programming 378 15.4 Longest common subsequence 390 15.5 Optimal binary search trees 397

16

Greedy Algorithms 414 16.1 An activity-selection problem 415 16.2 Elements of the greedy strategy 423 16.3 Huffman codes 428 16.4 Matroids and greedy methods 437 16.5 A task-scheduling problem as a matroid

? ? 17

Amortized Analysis 451 17.1 Aggregate analysis 452 17.2 The accounting method 456 17.3 The potential method 459 17.4 Dynamic tables 463

443

viii

Contents

V Advanced Data Structures Introduction 18

B-Trees 484 18.1 Deﬁnition of B-trees 488 18.2 Basic operations on B-trees 491 18.3 Deleting a key from a B-tree 499

19

Fibonacci Heaps 505 19.1 Structure of Fibonacci heaps 507 19.2 Mergeable-heap operations 510 19.3 Decreasing a key and deleting a node 518 19.4 Bounding the maximum degree 523

20

van Emde Boas Trees 531 20.1 Preliminary approaches 532 20.2 A recursive structure 536 20.3 The van Emde Boas tree 545

21

Data Structures for Disjoint Sets 561 21.1 Disjoint-set operations 561 21.2 Linked-list representation of disjoint sets 564 21.3 Disjoint-set forests 568 21.4 Analysis of union by rank with path compression

? VI

481

Graph Algorithms Introduction

587

22

Elementary Graph Algorithms 589 22.1 Representations of graphs 589 22.2 Breadth-ﬁrst search 594 22.3 Depth-ﬁrst search 603 22.4 Topological sort 612 22.5 Strongly connected components 615

23

Minimum Spanning Trees 624 23.1 Growing a minimum spanning tree 625 23.2 The algorithms of Kruskal and Prim 631

573

Contents

24

ix

Single-Source Shortest Paths 643 24.1 The Bellman-Ford algorithm 651 24.2 Single-source shortest paths in directed acyclic graphs 24.3 Dijkstra’s algorithm 658 24.4 Difference constraints and shortest paths 664 24.5 Proofs of shortest-paths properties 671

25

All-Pairs Shortest Paths 684 25.1 Shortest paths and matrix multiplication 686 25.2 The Floyd-Warshall algorithm 693 25.3 Johnson’s algorithm for sparse graphs 700

26

Maximum Flow 708 26.1 Flow networks 709 26.2 The Ford-Fulkerson method 714 26.3 Maximum bipartite matching 732 26.4 Push-relabel algorithms 736 26.5 The relabel-to-front algorithm 748

? ?

655

VII Selected Topics Introduction

769

27

Multithreaded Algorithms 772 27.1 The basics of dynamic multithreading 774 27.2 Multithreaded matrix multiplication 792 27.3 Multithreaded merge sort 797

28

Matrix Operations 813 28.1 Solving systems of linear equations 813 28.2 Inverting matrices 827 28.3 Symmetric positive-deﬁnite matrices and least-squares approximation 832

29

Linear Programming 843 29.1 Standard and slack forms 850 29.2 Formulating problems as linear programs 29.3 The simplex algorithm 864 29.4 Duality 879 29.5 The initial basic feasible solution 886

859

x

Contents

30

Polynomials and the FFT 898 30.1 Representing polynomials 900 30.2 The DFT and FFT 906 30.3 Efﬁcient FFT implementations 915

31

Number-Theoretic Algorithms 926 31.1 Elementary number-theoretic notions 927 31.2 Greatest common divisor 933 31.3 Modular arithmetic 939 31.4 Solving modular linear equations 946 31.5 The Chinese remainder theorem 950 31.6 Powers of an element 954 31.7 The RSA public-key cryptosystem 958 31.8 Primality testing 965 31.9 Integer factorization 975

? ? 32

? 33

String Matching 985 32.1 The naive string-matching algorithm 988 32.2 The Rabin-Karp algorithm 990 32.3 String matching with ﬁnite automata 995 32.4 The Knuth-Morris-Pratt algorithm 1002 Computational Geometry 1014 33.1 Line-segment properties 1015 33.2 Determining whether any pair of segments intersects 33.3 Finding the convex hull 1029 33.4 Finding the closest pair of points 1039

34

NP-Completeness 1048 34.1 Polynomial time 1053 34.2 Polynomial-time veriﬁcation 1061 34.3 NP-completeness and reducibility 1067 34.4 NP-completeness proofs 1078 34.5 NP-complete problems 1086

35

Approximation Algorithms 1106 35.1 The vertex-cover problem 1108 35.2 The traveling-salesman problem 1111 35.3 The set-covering problem 1117 35.4 Randomization and linear programming 35.5 The subset-sum problem 1128

1123

1021

Contents

xi

VIII Appendix: Mathematical Background Introduction A

1143

Summations 1145 A.1 Summation formulas and properties A.2 Bounding summations 1149

1145

B

Sets, Etc. 1158 B.1 Sets 1158 B.2 Relations 1163 B.3 Functions 1166 B.4 Graphs 1168 B.5 Trees 1173

C

Counting and Probability 1183 C.1 Counting 1183 C.2 Probability 1189 C.3 Discrete random variables 1196 C.4 The geometric and binomial distributions 1201 C.5 The tails of the binomial distribution 1208

? D

Matrices 1217 D.1 Matrices and matrix operations D.2 Basic matrix properties 1222 Bibliography Index

1251

1231

1217

Preface

Before there were computers, there were algorithms. But now that there are computers, there are even more algorithms, and algorithms lie at the heart of computing. This book provides a comprehensive introduction to the modern study of computer algorithms. It presents many algorithms and covers them in considerable depth, yet makes their design and analysis accessible to all levels of readers. We have tried to keep explanations elementary without sacriﬁcing depth of coverage or mathematical rigor. Each chapter presents an algorithm, a design technique, an application area, or a related topic. Algorithms are described in English and in a pseudocode designed to be readable by anyone who has done a little programming. The book contains 244 ﬁgures—many with multiple parts—illustrating how the algorithms work. Since we emphasize efﬁciency as a design criterion, we include careful analyses of the running times of all our algorithms. The text is intended primarily for use in undergraduate or graduate courses in algorithms or data structures. Because it discusses engineering issues in algorithm design, as well as mathematical aspects, it is equally well suited for self-study by technical professionals. In this, the third edition, we have once again updated the entire book. The changes cover a broad spectrum, including new chapters, revised pseudocode, and a more active writing style. To the teacher We have designed this book to be both versatile and complete. You should ﬁnd it useful for a variety of courses, from an und...

INTRODUCTION TO

ALGORITHMS T H I R D

E D I T I O N

Introduction to Algorithms Third Edition

Thomas H. Cormen Charles E. Leiserson Ronald L. Rivest Clifford Stein

Introduction to Algorithms Third Edition

The MIT Press Cambridge, Massachusetts

London, England

c 2009 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form or by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher. For information about special quantity discounts, please email special [email protected] This book was set in Times Roman and Mathtime Pro 2 by the authors. Printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Introduction to algorithms / Thomas H. Cormen . . . [et al.].—3rd ed. p. cm. Includes bibliographical references and index. ISBN 978-0-262-03384-8 (hardcover : alk. paper)—ISBN 978-0-262-53305-8 (pbk. : alk. paper) 1. Computer programming. 2. Computer algorithms. I. Cormen, Thomas H. QA76.6.I5858 2009 005.1—dc22 2009008593 10 9 8 7 6 5 4 3 2

Contents

Preface

xiii

I Foundations Introduction

3

1

The Role of Algorithms in Computing 5 1.1 Algorithms 5 1.2 Algorithms as a technology 11

2

Getting Started 16 2.1 Insertion sort 16 2.2 Analyzing algorithms 23 2.3 Designing algorithms 29

3

Growth of Functions 43 3.1 Asymptotic notation 43 3.2 Standard notations and common functions

4

? 5

?

53

Divide-and-Conquer 65 4.1 The maximum-subarray problem 68 4.2 Strassen’s algorithm for matrix multiplication 75 4.3 The substitution method for solving recurrences 83 4.4 The recursion-tree method for solving recurrences 88 4.5 The master method for solving recurrences 93 4.6 Proof of the master theorem 97 Probabilistic Analysis and Randomized Algorithms 114 5.1 The hiring problem 114 5.2 Indicator random variables 118 5.3 Randomized algorithms 122 5.4 Probabilistic analysis and further uses of indicator random variables 130

vi

Contents

II Sorting and Order Statistics Introduction 6

7

8

9

147

Heapsort 151 6.1 Heaps 151 6.2 Maintaining the heap property 6.3 Building a heap 156 6.4 The heapsort algorithm 159 6.5 Priority queues 162

154

Quicksort 170 7.1 Description of quicksort 170 7.2 Performance of quicksort 174 7.3 A randomized version of quicksort 7.4 Analysis of quicksort 180 Sorting in Linear Time 191 8.1 Lower bounds for sorting 8.2 Counting sort 194 8.3 Radix sort 197 8.4 Bucket sort 200

179

191

Medians and Order Statistics 213 9.1 Minimum and maximum 214 9.2 Selection in expected linear time 215 9.3 Selection in worst-case linear time 220

III Data Structures Introduction 10

11

?

229

Elementary Data Structures 232 10.1 Stacks and queues 232 10.2 Linked lists 236 10.3 Implementing pointers and objects 10.4 Representing rooted trees 246 Hash Tables 253 11.1 Direct-address tables 254 11.2 Hash tables 256 11.3 Hash functions 262 11.4 Open addressing 269 11.5 Perfect hashing 277

241

Contents

12

? 13

14

vii

Binary Search Trees 286 12.1 What is a binary search tree? 286 12.2 Querying a binary search tree 289 12.3 Insertion and deletion 294 12.4 Randomly built binary search trees 299 Red-Black Trees 308 13.1 Properties of red-black trees 13.2 Rotations 312 13.3 Insertion 315 13.4 Deletion 323

308

Augmenting Data Structures 339 14.1 Dynamic order statistics 339 14.2 How to augment a data structure 14.3 Interval trees 348

345

IV Advanced Design and Analysis Techniques Introduction

357

15

Dynamic Programming 359 15.1 Rod cutting 360 15.2 Matrix-chain multiplication 370 15.3 Elements of dynamic programming 378 15.4 Longest common subsequence 390 15.5 Optimal binary search trees 397

16

Greedy Algorithms 414 16.1 An activity-selection problem 415 16.2 Elements of the greedy strategy 423 16.3 Huffman codes 428 16.4 Matroids and greedy methods 437 16.5 A task-scheduling problem as a matroid

? ? 17

Amortized Analysis 451 17.1 Aggregate analysis 452 17.2 The accounting method 456 17.3 The potential method 459 17.4 Dynamic tables 463

443

viii

Contents

V Advanced Data Structures Introduction 18

B-Trees 484 18.1 Deﬁnition of B-trees 488 18.2 Basic operations on B-trees 491 18.3 Deleting a key from a B-tree 499

19

Fibonacci Heaps 505 19.1 Structure of Fibonacci heaps 507 19.2 Mergeable-heap operations 510 19.3 Decreasing a key and deleting a node 518 19.4 Bounding the maximum degree 523

20

van Emde Boas Trees 531 20.1 Preliminary approaches 532 20.2 A recursive structure 536 20.3 The van Emde Boas tree 545

21

Data Structures for Disjoint Sets 561 21.1 Disjoint-set operations 561 21.2 Linked-list representation of disjoint sets 564 21.3 Disjoint-set forests 568 21.4 Analysis of union by rank with path compression

? VI

481

Graph Algorithms Introduction

587

22

Elementary Graph Algorithms 589 22.1 Representations of graphs 589 22.2 Breadth-ﬁrst search 594 22.3 Depth-ﬁrst search 603 22.4 Topological sort 612 22.5 Strongly connected components 615

23

Minimum Spanning Trees 624 23.1 Growing a minimum spanning tree 625 23.2 The algorithms of Kruskal and Prim 631

573

Contents

24

ix

Single-Source Shortest Paths 643 24.1 The Bellman-Ford algorithm 651 24.2 Single-source shortest paths in directed acyclic graphs 24.3 Dijkstra’s algorithm 658 24.4 Difference constraints and shortest paths 664 24.5 Proofs of shortest-paths properties 671

25

All-Pairs Shortest Paths 684 25.1 Shortest paths and matrix multiplication 686 25.2 The Floyd-Warshall algorithm 693 25.3 Johnson’s algorithm for sparse graphs 700

26

Maximum Flow 708 26.1 Flow networks 709 26.2 The Ford-Fulkerson method 714 26.3 Maximum bipartite matching 732 26.4 Push-relabel algorithms 736 26.5 The relabel-to-front algorithm 748

? ?

655

VII Selected Topics Introduction

769

27

Multithreaded Algorithms 772 27.1 The basics of dynamic multithreading 774 27.2 Multithreaded matrix multiplication 792 27.3 Multithreaded merge sort 797

28

Matrix Operations 813 28.1 Solving systems of linear equations 813 28.2 Inverting matrices 827 28.3 Symmetric positive-deﬁnite matrices and least-squares approximation 832

29

Linear Programming 843 29.1 Standard and slack forms 850 29.2 Formulating problems as linear programs 29.3 The simplex algorithm 864 29.4 Duality 879 29.5 The initial basic feasible solution 886

859

x

Contents

30

Polynomials and the FFT 898 30.1 Representing polynomials 900 30.2 The DFT and FFT 906 30.3 Efﬁcient FFT implementations 915

31

Number-Theoretic Algorithms 926 31.1 Elementary number-theoretic notions 927 31.2 Greatest common divisor 933 31.3 Modular arithmetic 939 31.4 Solving modular linear equations 946 31.5 The Chinese remainder theorem 950 31.6 Powers of an element 954 31.7 The RSA public-key cryptosystem 958 31.8 Primality testing 965 31.9 Integer factorization 975

? ? 32

? 33

String Matching 985 32.1 The naive string-matching algorithm 988 32.2 The Rabin-Karp algorithm 990 32.3 String matching with ﬁnite automata 995 32.4 The Knuth-Morris-Pratt algorithm 1002 Computational Geometry 1014 33.1 Line-segment properties 1015 33.2 Determining whether any pair of segments intersects 33.3 Finding the convex hull 1029 33.4 Finding the closest pair of points 1039

34

NP-Completeness 1048 34.1 Polynomial time 1053 34.2 Polynomial-time veriﬁcation 1061 34.3 NP-completeness and reducibility 1067 34.4 NP-completeness proofs 1078 34.5 NP-complete problems 1086

35

Approximation Algorithms 1106 35.1 The vertex-cover problem 1108 35.2 The traveling-salesman problem 1111 35.3 The set-covering problem 1117 35.4 Randomization and linear programming 35.5 The subset-sum problem 1128

1123

1021

Contents

xi

VIII Appendix: Mathematical Background Introduction A

1143

Summations 1145 A.1 Summation formulas and properties A.2 Bounding summations 1149

1145

B

Sets, Etc. 1158 B.1 Sets 1158 B.2 Relations 1163 B.3 Functions 1166 B.4 Graphs 1168 B.5 Trees 1173

C

Counting and Probability 1183 C.1 Counting 1183 C.2 Probability 1189 C.3 Discrete random variables 1196 C.4 The geometric and binomial distributions 1201 C.5 The tails of the binomial distribution 1208

? D

Matrices 1217 D.1 Matrices and matrix operations D.2 Basic matrix properties 1222 Bibliography Index

1251

1231

1217

Preface

Before there were computers, there were algorithms. But now that there are computers, there are even more algorithms, and algorithms lie at the heart of computing. This book provides a comprehensive introduction to the modern study of computer algorithms. It presents many algorithms and covers them in considerable depth, yet makes their design and analysis accessible to all levels of readers. We have tried to keep explanations elementary without sacriﬁcing depth of coverage or mathematical rigor. Each chapter presents an algorithm, a design technique, an application area, or a related topic. Algorithms are described in English and in a pseudocode designed to be readable by anyone who has done a little programming. The book contains 244 ﬁgures—many with multiple parts—illustrating how the algorithms work. Since we emphasize efﬁciency as a design criterion, we include careful analyses of the running times of all our algorithms. The text is intended primarily for use in undergraduate or graduate courses in algorithms or data structures. Because it discusses engineering issues in algorithm design, as well as mathematical aspects, it is equally well suited for self-study by technical professionals. In this, the third edition, we have once again updated the entire book. The changes cover a broad spectrum, including new chapters, revised pseudocode, and a more active writing style. To the teacher We have designed this book to be both versatile and complete. You should ﬁnd it useful for a variety of courses, from an und...

*When life gives you a hundred reasons to cry, show life that you have a thousand reasons to smile*

© Copyright 2015 - 2020 ZDOC.TIPS - All rights reserved.