bpo-36876: Fix the C analyzer tool. (GH-22841)

The original tool wasn't working right and it was simpler to create a new one, partially re-using some of the old code. At this point the tool runs properly on the master. (Try: ./python Tools/c-analyzer/c-analyzer.py analyze.)  It take ~40 seconds on my machine to analyze the full CPython code base.

Note that we'll need to iron out some OS-specific stuff (e.g. preprocessor). We're okay though since this tool isn't used yet in our workflow. We will also need to verify the analysis results in detail before activating the check in CI, though I'm pretty sure it's close.

https://bugs.python.org/issue36876

1 files changed, 48 insertions, 0 deletions

diff --git a/Tools/c-analyzer/c_common/iterutil.py b/Tools/c-analyzer/c_common/iterutil.py
new file mode 100644
index 00000000000..6ded105304e
--- /dev/null
+++ b/Tools/c-analyzer/c_common/iterutil.py
@@ -0,0 +1,48 @@
+
+_NOT_SET = object()
+
+
+def peek_and_iter(items):
+    if not items:
+        return None, None
+    items = iter(items)
+    try:
+        peeked = next(items)
+    except StopIteration:
+        return None, None
+    def chain():
+        yield peeked
+        yield from items
+    return chain(), peeked
+
+
+def iter_many(items, onempty=None):
+    if not items:
+        if onempty is None:
+            return
+        if not callable(onempty):
+            raise onEmpty
+        items = onempty(items)
+        yield from iter_many(items, onempty=None)
+        return
+    items = iter(items)
+    try:
+        first = next(items)
+    except StopIteration:
+        if onempty is None:
+            return
+        if not callable(onempty):
+            raise onEmpty
+        items = onempty(items)
+        yield from iter_many(items, onempty=None)
+    else:
+        try:
+            second = next(items)
+        except StopIteration:
+            yield first, False
+            return
+        else:
+            yield first, True
+            yield second, True
+        for item in items:
+            yield item, True