C++ – 默默的点滴

Clang的线程安全分析模块Thread Safety Analysis

背景

最近在浏览 Android 11 源代码的时候，发现在ART虚拟机头文件 art_method.h 中，存在大量的类似REQUIRES_SHARED(Locks::mutator_lock_); 的代码。

继续阅读Clang的线程安全分析模块Thread Safety Analysis

Parsing C++ in Python with Clang

macOS Mojave 10.14.2 llvm 7.0.1

$ brew install llvm --with-clang --with-python --with-lld

$ export DYLD_LIBRARY_PATH=/usr/local/opt/llvm/lib

$ export LLVM_VER=$(brew list --versions | grep llvm | sed 's/\([0-9]\).*/\1/g' | tr -cd "[0-9]")

$ pip install clang

$ brew install llvm --with-clang --with-python --with-lld

$ export DYLD_LIBRARY_PATH=/usr/local/opt/llvm/lib

$ export LLVM_VER=$(brew list --versions | grep llvm | sed 's/$[0-9]$.*/\1/g' | tr -cd "[0-9]")

$ pip install clang

#!/usr/bin/env python
# -*- coding: utf-8 -*-

""" Usage: call with <filename> <typename>
"""

import sys
import clang.cindex

# 也可以使用如下命令指定libclang的加载位置,这个动态库可以直接拷贝到其他机器上使用，是个独立库
# clang.cindex.Config.set_library_path('/usr/local/opt/llvm/lib')

# 如下可以简化代码
# from clang.cindex import CursorKind
# from clang.cindex import AccessSpecifier

def find_typerefs(node, typename):
    """ Find all references to the type named 'typename'
    """
    if node.kind.is_reference():
        ref_node = node.get_definition()
        if ref_node :
            if ref_node.spelling == typename:
                print 'Found %s [line=%s, col=%s]' % (
                    typename, node.location.line, node.location.column)
    # Recurse for children of this node
    for c in node.get_children():
        find_typerefs(c, typename)

index = clang.cindex.Index.create()
tu = index.parse(sys.argv[1])
print 'Translation unit:', tu.spelling
find_typerefs(tu.cursor, sys.argv[2])

#!/usr/bin/env python

# -*- coding: utf-8 -*-

""" Usage: call with <filename> <typename>

"""

import sys

import clang.cindex

# 也可以使用如下命令指定libclang的加载位置,这个动态库可以直接拷贝到其他机器上使用，是个独立库

# clang.cindex.Config.set_library_path('/usr/local/opt/llvm/lib')

# 如下可以简化代码

# from clang.cindex import CursorKind

# from clang.cindex import AccessSpecifier

def find_typerefs(node, typename):

""" Find all references to the type named 'typename'

"""

if node.kind.is_reference():

ref_node = node.get_definition()

if ref_node :

if ref_node.spelling == typename:

print 'Found %s [line=%s, col=%s]' % (

typename, node.location.line, node.location.column)

# Recurse for children of this node

for c in node.get_children():

find_typerefs(c, typename)

index = clang.cindex.Index.create()

tu = index.parse(sys.argv[1])

print 'Translation unit:', tu.spelling

find_typerefs(tu.cursor, sys.argv[2])

#!/usr/bin/env python
# -*- coding: utf-8 -*-
""" C++ linter using libclang. Call with [filenames] """
from __future__ import print_function

from clang.cindex import Config, TypeKind, CursorKind, Index
from pprint import pprint
import platform
import sys

if platform.system() == "Linux":
    Config.set_library_file("/usr/lib/llvm-3.4/lib/libclang.so")
else:
    Config.set_library_file("/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libclang.dylib")


def error(out_errors, node, msg):
    error_msg = "{}:{}:  {}".format(node.location.file, node.location.line, msg)
    out_errors.append(error_msg)
    print(error_msg, file=sys.stderr)


def is_mut_ref(arg_type):
    if arg_type.kind in [TypeKind.POINTER, TypeKind.LVALUEREFERENCE,
                         TypeKind.INCOMPLETEARRAY, TypeKind.CONSTANTARRAY]:

        if arg_type.kind in [TypeKind.POINTER, TypeKind.LVALUEREFERENCE]:
            pointee_type = arg_type.get_pointee()
        else:
            pointee_type = arg_type.get_array_element_type()

        # print("pointee_type.kind: {}".format(pointee_type.kind))
        # print("pointee_type.is_const_qualified(): {}".format(pointee_type.is_const_qualified()))

        if not pointee_type.is_const_qualified():
            return True

    return False


def check_argument(out_errors, function, node, function_parse_progress):
    assert node.kind == CursorKind.PARM_DECL

    if function.kind == CursorKind.FUNCTION_DECL and function.spelling == "main":
        # Ignore main function
        return

    # print("")
    # print("node.spelling: {}".format(node.spelling))
    # print("node.type.kind: {}".format(node.type.kind))
    # print("node.type.get_ref_qualifier(): {}".format(node.type.get_ref_qualifier()))
    # print("node.type.is_const_qualified(): {}".format(node.type.is_const_qualified()))
    # pprint(dir(node))

    name = node.spelling
    if not name:
        # Ignore nameless arguments (e.g. Foo(Foo&))
        return

    if is_mut_ref(node.type):
        if name.startswith("o_"):
            function_parse_progress["state"] = "out"
        elif name.startswith("io_"):
            if function_parse_progress["state"] != "io":
                error(out_errors, node, "io_ arguments should be first")
            function_parse_progress["state"] = "io"

        else:
            error(out_errors, node,
                  "Non-const reference/pointer/array argument should be prefixed with " \
                  "either o_ (for out) or io_ (for in-out), e.g. 'o_{}'".format(name))
    else:
        if function_parse_progress["state"] == "out":
            error(out_errors, node, "input arguments should come before output arguments")
        function_parse_progress["state"] = "in"


def do_lint(out_errors, node, root_file):
    if node.location.file and node.location.file.name != root_file.name:
        # This is ugly, but works.
        return

    # print("{}:{} node.kind: {}".format(node.location.file, node.location.line, node.kind))
    # print("node.translation_unit: {}".format(node.translation_unit))
    # # pprint(dir(node.translation_unit))
    # print("node.location.file: {}".format(node.location.file))
    # pprint(dir(node.location))
    # exit()

    # CursorKind.CONSTRUCTOR excluded: references there are often stored, so not o_ or io_
    if node.kind in [CursorKind.FUNCTION_DECL, CursorKind.CXX_METHOD]:
        # print("Found a function!")
        # print("node.spelling: {}".format(node.spelling))
        # print("node.displayname: {}".format(node.displayname))

        function_parse_progress = {
            "state": "io", # "io", "in" or "out"
        }

        for arg in node.get_arguments():
            check_argument(out_errors, node, arg, function_parse_progress)

    # Recurse for children of this node
    for c in node.get_children():
        do_lint(out_errors, c, root_file)


def lint_file(filepath):
    index = Index.create()
    tu = index.parse(filepath)
    root_file = tu.get_file(tu.spelling)

    errors = []
    do_lint(errors, tu.cursor, root_file)
    return errors


def main():
    if len(sys.argv) == 1:
        print("Usage: {} [filenames]".format(sys.argv[0]))
        return

    for filepath in sys.argv[1:]:
        lint_file(filepath)


if __name__ == "__main__":
    main()

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

#!/usr/bin/env python

# -*- coding: utf-8 -*-

""" C++ linter using libclang. Call with [filenames] """

from __future__ import print_function

from clang.cindex import Config, TypeKind, CursorKind, Index

from pprint import pprint

import platform

import sys

if platform.system() == "Linux":

Config.set_library_file("/usr/lib/llvm-3.4/lib/libclang.so")

else:

Config.set_library_file("/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libclang.dylib")

def error(out_errors, node, msg):

error_msg = "{}:{}: {}".format(node.location.file, node.location.line, msg)

out_errors.append(error_msg)

print(error_msg, file=sys.stderr)

def is_mut_ref(arg_type):

if arg_type.kind in [TypeKind.POINTER, TypeKind.LVALUEREFERENCE,

TypeKind.INCOMPLETEARRAY, TypeKind.CONSTANTARRAY]:

if arg_type.kind in [TypeKind.POINTER, TypeKind.LVALUEREFERENCE]:

pointee_type = arg_type.get_pointee()

else:

pointee_type = arg_type.get_array_element_type()

# print("pointee_type.kind: {}".format(pointee_type.kind))

# print("pointee_type.is_const_qualified(): {}".format(pointee_type.is_const_qualified()))

if not pointee_type.is_const_qualified():

return True

return False

def check_argument(out_errors, function, node, function_parse_progress):

assert node.kind == CursorKind.PARM_DECL

if function.kind == CursorKind.FUNCTION_DECL and function.spelling == "main":

# Ignore main function

return

# print("")

# print("node.spelling: {}".format(node.spelling))

# print("node.type.kind: {}".format(node.type.kind))

# print("node.type.get_ref_qualifier(): {}".format(node.type.get_ref_qualifier()))

# print("node.type.is_const_qualified(): {}".format(node.type.is_const_qualified()))

# pprint(dir(node))

name = node.spelling

if not name:

# Ignore nameless arguments (e.g. Foo(Foo&))

return

if is_mut_ref(node.type):

if name.startswith("o_"):

function_parse_progress["state"] = "out"

elif name.startswith("io_"):

if function_parse_progress["state"] != "io":

error(out_errors, node, "io_ arguments should be first")

function_parse_progress["state"] = "io"

else:

error(out_errors, node,

"Non-const reference/pointer/array argument should be prefixed with " \

"either o_ (for out) or io_ (for in-out), e.g. 'o_{}'".format(name))

else:

if function_parse_progress["state"] == "out":

error(out_errors, node, "input arguments should come before output arguments")

function_parse_progress["state"] = "in"

def do_lint(out_errors, node, root_file):

if node.location.file and node.location.file.name != root_file.name:

# This is ugly, but works.

return

# print("{}:{} node.kind: {}".format(node.location.file, node.location.line, node.kind))

# print("node.translation_unit: {}".format(node.translation_unit))

# # pprint(dir(node.translation_unit))

# print("node.location.file: {}".format(node.location.file))

# pprint(dir(node.location))

# exit()

# CursorKind.CONSTRUCTOR excluded: references there are often stored, so not o_ or io_

if node.kind in [CursorKind.FUNCTION_DECL, CursorKind.CXX_METHOD]:

# print("Found a function!")

# print("node.spelling: {}".format(node.spelling))

# print("node.displayname: {}".format(node.displayname))

function_parse_progress = {

"state": "io", # "io", "in" or "out"

}

for arg in node.get_arguments():

check_argument(out_errors, node, arg, function_parse_progress)

# Recurse for children of this node

for c in node.get_children():

do_lint(out_errors, c, root_file)

def lint_file(filepath):

index = Index.create()

tu = index.parse(filepath)

root_file = tu.get_file(tu.spelling)

errors = []

do_lint(errors, tu.cursor, root_file)

return errors

def main():

if len(sys.argv) == 1:

print("Usage: {} [filenames]".format(sys.argv[0]))

return

for filepath in sys.argv[1:]:

lint_file(filepath)

if __name__ == "__main__":

main()

#!/usr/bin/env python

import clang.cindex
import sys
import os
import logging
import argparse
import fnmatch

from DotGenerator import *

index = clang.cindex.Index.create()
dotGenerator = DotGenerator()


def findFilesInDir(rootDir, patterns):
    """ Searches for files in rootDir which file names mathes the given pattern. Returns
    a list of file paths of found files"""
    foundFiles = []
    for root, dirs, files in os.walk(rootDir):
        for p in patterns:
            for filename in fnmatch.filter(files, p):
                foundFiles.append(os.path.join(root, filename))
    return foundFiles


def processClassField(cursor):
    """ Returns the name and the type of the given class field.
    The cursor must be of kind CursorKind.FIELD_DECL"""
    type = None
    fieldChilds = list(cursor.get_children())
    if len(fieldChilds) == 0:  # if there are not cursorchildren, the type is some primitive datatype
        type = cursor.type.spelling
    else:  # if there are cursorchildren, the type is some non-primitive datatype (a class or class template)
        for cc in fieldChilds:
            if cc.kind == clang.cindex.CursorKind.TEMPLATE_REF:
                type = cc.spelling
            elif cc.kind == clang.cindex.CursorKind.TYPE_REF:
                type = cursor.type.spelling
    name = cursor.spelling
    return name, type


def processClassMemberDeclaration(umlClass, cursor):
    """ Processes a cursor corresponding to a class member declaration and
    appends the extracted information to the given umlClass """
    if cursor.kind == clang.cindex.CursorKind.CXX_BASE_SPECIFIER:
        for baseClass in cursor.get_children():
            if baseClass.kind == clang.cindex.CursorKind.TEMPLATE_REF:
                umlClass.parents.append(baseClass.spelling)
            elif baseClass.kind == clang.cindex.CursorKind.TYPE_REF:
                umlClass.parents.append(baseClass.type.spelling)
    elif cursor.kind == clang.cindex.CursorKind.FIELD_DECL:  # non static data member
        name, type = processClassField(cursor)
        if name is not None and type is not None:
            # clang < 3.5: needs patched cindex.py to have
            # clang.cindex.AccessSpecifier available:
            # https://gitorious.org/clang-mirror/clang-mirror/commit/e3d4e7c9a45ed9ad4645e4dc9f4d3b4109389cb7
            if cursor.access_specifier == clang.cindex.AccessSpecifier.PUBLIC:
                umlClass.publicFields.append((name, type))
            elif cursor.access_specifier == clang.cindex.AccessSpecifier.PRIVATE:
                umlClass.privateFields.append((name, type))
            elif cursor.access_specifier == clang.cindex.AccessSpecifier.PROTECTED:
                umlClass.protectedFields.append((name, type))
    elif cursor.kind == clang.cindex.CursorKind.CXX_METHOD:
        try:
            returnType, argumentTypes = cursor.type.spelling.split(' ', 1)
            if cursor.access_specifier == clang.cindex.AccessSpecifier.PUBLIC:
                umlClass.publicMethods.append((returnType, cursor.spelling, argumentTypes))
            elif cursor.access_specifier == clang.cindex.AccessSpecifier.PRIVATE:
                umlClass.privateMethods.append((returnType, cursor.spelling, argumentTypes))
            elif cursor.access_specifier == clang.cindex.AccessSpecifier.PROTECTED:
                umlClass.protectedMethods.append((returnType, cursor.spelling, argumentTypes))
        except:
            logging.error("Invalid CXX_METHOD declaration! " + str(cursor.type.spelling))
    elif cursor.kind == clang.cindex.CursorKind.FUNCTION_TEMPLATE:
        returnType, argumentTypes = cursor.type.spelling.split(' ', 1)
        if cursor.access_specifier == clang.cindex.AccessSpecifier.PUBLIC:
            umlClass.publicMethods.append((returnType, cursor.spelling, argumentTypes))
        elif cursor.access_specifier == clang.cindex.AccessSpecifier.PRIVATE:
            umlClass.privateMethods.append((returnType, cursor.spelling, argumentTypes))
        elif cursor.access_specifier == clang.cindex.AccessSpecifier.PROTECTED:
            umlClass.protectedMethods.append((returnType, cursor.spelling, argumentTypes))


def processClass(cursor, inclusionConfig):
    """ Processes an ast node that is a class. """
    umlClass = UmlClass()  # umlClass is the datastructure for the DotGenerator
                           # that stores the necessary information about a single class.
                           # We extract this information from the clang ast hereafter ...
    if cursor.kind == clang.cindex.CursorKind.CLASS_TEMPLATE:
        # process declarations like:
        #   template <typename T> class MyClass
        umlClass.fqn = cursor.spelling
    else:
        # process declarations like:
        #   class MyClass ...
        #   struct MyStruct ...
        umlClass.fqn = cursor.type.spelling  # the fully qualified name

    import re
    if (inclusionConfig['excludeClasses'] and
            re.match(inclusionConfig['excludeClasses'], umlClass.fqn)):
        return

    if (inclusionConfig['includeClasses'] and not
            re.match(inclusionConfig['includeClasses'], umlClass.fqn)):
        return

    for c in cursor.get_children():
        # process member variables and methods declarations
        processClassMemberDeclaration(umlClass, c)

    dotGenerator.addClass(umlClass)


def traverseAst(cursor, inclusionConfig):
    if (cursor.kind == clang.cindex.CursorKind.CLASS_DECL
            or cursor.kind == clang.cindex.CursorKind.STRUCT_DECL
            or cursor.kind == clang.cindex.CursorKind.CLASS_TEMPLATE):
        # if the current cursor is a class, class template or struct declaration,
        # we process it further ...
        processClass(cursor, inclusionConfig)
    for child_node in cursor.get_children():
        traverseAst(child_node, inclusionConfig)


def parseTranslationUnit(filePath, includeDirs, inclusionConfig):
    clangArgs = ['-x', 'c++'] + ['-I' + includeDir for includeDir in includeDirs]
    tu = index.parse(filePath, args=clangArgs, options=clang.cindex.TranslationUnit.PARSE_SKIP_FUNCTION_BODIES)
    for diagnostic in tu.diagnostics:
        logging.debug(diagnostic)
    logging.info('Translation unit:' + tu.spelling + "\n")
    traverseAst(tu.cursor, inclusionConfig)


if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="CodeDependencyVisualizer (CDV)")
    parser.add_argument('-d', required=True, help="directory with source files to parse (searches recusively)")
    parser.add_argument('-o', '--outFile', default='uml.dot', help="output file name / name of generated dot file")
    parser.add_argument('-u', '--withUnusedHeaders', help="parse unused header files (slow)")
    parser.add_argument('-a', '--associations', action="store_true", help="draw class member assiciations")
    parser.add_argument('-i', '--inheritances', action="store_true", help="draw class inheritances")
    parser.add_argument('-p', '--privMembers', action="store_true", help="show private members")
    parser.add_argument('-t', '--protMembers', action="store_true", help="show protected members")
    parser.add_argument('-P', '--pubMembers', action="store_true", help="show public members")
    parser.add_argument('-I', '--includeDirs', help="additional search path(s) for include files (seperated by space)", nargs='+')
    parser.add_argument('-v', '--verbose', action="store_true", help="print verbose information for debugging purposes")
    parser.add_argument('--excludeClasses', help="classes matching this pattern will be excluded")
    parser.add_argument('--includeClasses', help="only classes matching this pattern will be included")

    args = vars(parser.parse_args(sys.argv[1:]))

    filesToParsePatterns = ['*.cpp', '*.cxx', '*.c', '*.cc']
    if args['withUnusedHeaders']:
        filesToParsePatterns += ['*.h', '*.hxx', '*.hpp']
    filesToParse = findFilesInDir(args['d'], filesToParsePatterns)
    subdirectories = [x[0] for x in os.walk(args['d'])]

    loggingFormat = "%(levelname)s - %(module)s: %(message)s"
    logging.basicConfig(format=loggingFormat, level=logging.INFO)
    if args['verbose']:
        logging.basicConfig(format=loggingFormat, level=logging.DEBUG)

    logging.info("found " + str(len(filesToParse)) + " source files.")

    for sourceFile in filesToParse:
        logging.info("parsing file " + sourceFile)
        parseTranslationUnit(sourceFile, args['includeDirs'], {
            'excludeClasses': args['excludeClasses'],
            'includeClasses': args['includeClasses']})

    dotGenerator.setDrawAssociations(args['associations'])
    dotGenerator.setDrawInheritances(args['inheritances'])
    dotGenerator.setShowPrivMethods(args['privMembers'])
    dotGenerator.setShowProtMethods(args['protMembers'])
    dotGenerator.setShowPubMethods(args['pubMembers'])

    dotfileName = args['outFile']
    logging.info("generating dotfile " + dotfileName)
    with open(dotfileName, 'w') as dotfile:
        dotfile.write(dotGenerator.generate())

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

#!/usr/bin/env python

import clang.cindex

import sys

import os

import logging

import argparse

import fnmatch

from DotGenerator import *

index = clang.cindex.Index.create()

dotGenerator = DotGenerator()

def findFilesInDir(rootDir, patterns):

""" Searches for files in rootDir which file names mathes the given pattern. Returns

a list of file paths of found files"""

foundFiles = []

for root, dirs, files in os.walk(rootDir):

for p in patterns:

for filename in fnmatch.filter(files, p):

foundFiles.append(os.path.join(root, filename))

return foundFiles

def processClassField(cursor):

""" Returns the name and the type of the given class field.

The cursor must be of kind CursorKind.FIELD_DECL"""

type = None

fieldChilds = list(cursor.get_children())

if len(fieldChilds) == 0: # if there are not cursorchildren, the type is some primitive datatype

type = cursor.type.spelling

else: # if there are cursorchildren, the type is some non-primitive datatype (a class or class template)

for cc in fieldChilds:

if cc.kind == clang.cindex.CursorKind.TEMPLATE_REF:

type = cc.spelling

elif cc.kind == clang.cindex.CursorKind.TYPE_REF:

type = cursor.type.spelling

name = cursor.spelling

return name, type

def processClassMemberDeclaration(umlClass, cursor):

""" Processes a cursor corresponding to a class member declaration and

appends the extracted information to the given umlClass """

if cursor.kind == clang.cindex.CursorKind.CXX_BASE_SPECIFIER:

for baseClass in cursor.get_children():

if baseClass.kind == clang.cindex.CursorKind.TEMPLATE_REF:

umlClass.parents.append(baseClass.spelling)

elif baseClass.kind == clang.cindex.CursorKind.TYPE_REF:

umlClass.parents.append(baseClass.type.spelling)

elif cursor.kind == clang.cindex.CursorKind.FIELD_DECL: # non static data member

name, type = processClassField(cursor)

if name is not None and type is not None:

# clang < 3.5: needs patched cindex.py to have

# clang.cindex.AccessSpecifier available:

# https://gitorious.org/clang-mirror/clang-mirror/commit/e3d4e7c9a45ed9ad4645e4dc9f4d3b4109389cb7

if cursor.access_specifier == clang.cindex.AccessSpecifier.PUBLIC:

umlClass.publicFields.append((name, type))

elif cursor.access_specifier == clang.cindex.AccessSpecifier.PRIVATE:

umlClass.privateFields.append((name, type))

elif cursor.access_specifier == clang.cindex.AccessSpecifier.PROTECTED:

umlClass.protectedFields.append((name, type))

elif cursor.kind == clang.cindex.CursorKind.CXX_METHOD:

try:

returnType, argumentTypes = cursor.type.spelling.split(' ', 1)

if cursor.access_specifier == clang.cindex.AccessSpecifier.PUBLIC:

umlClass.publicMethods.append((returnType, cursor.spelling, argumentTypes))

elif cursor.access_specifier == clang.cindex.AccessSpecifier.PRIVATE:

umlClass.privateMethods.append((returnType, cursor.spelling, argumentTypes))

elif cursor.access_specifier == clang.cindex.AccessSpecifier.PROTECTED:

umlClass.protectedMethods.append((returnType, cursor.spelling, argumentTypes))

except:

logging.error("Invalid CXX_METHOD declaration! " + str(cursor.type.spelling))

elif cursor.kind == clang.cindex.CursorKind.FUNCTION_TEMPLATE:

returnType, argumentTypes = cursor.type.spelling.split(' ', 1)

if cursor.access_specifier == clang.cindex.AccessSpecifier.PUBLIC:

umlClass.publicMethods.append((returnType, cursor.spelling, argumentTypes))

elif cursor.access_specifier == clang.cindex.AccessSpecifier.PRIVATE:

umlClass.privateMethods.append((returnType, cursor.spelling, argumentTypes))

elif cursor.access_specifier == clang.cindex.AccessSpecifier.PROTECTED:

umlClass.protectedMethods.append((returnType, cursor.spelling, argumentTypes))

def processClass(cursor, inclusionConfig):

""" Processes an ast node that is a class. """

umlClass = UmlClass() # umlClass is the datastructure for the DotGenerator

# that stores the necessary information about a single class.

# We extract this information from the clang ast hereafter ...

if cursor.kind == clang.cindex.CursorKind.CLASS_TEMPLATE:

# process declarations like:

# template <typename T> class MyClass

umlClass.fqn = cursor.spelling

else:

# process declarations like:

# class MyClass ...

# struct MyStruct ...

umlClass.fqn = cursor.type.spelling # the fully qualified name

import re

if (inclusionConfig['excludeClasses'] and

re.match(inclusionConfig['excludeClasses'], umlClass.fqn)):

return

if (inclusionConfig['includeClasses'] and not

re.match(inclusionConfig['includeClasses'], umlClass.fqn)):

return

for c in cursor.get_children():

# process member variables and methods declarations

processClassMemberDeclaration(umlClass, c)

dotGenerator.addClass(umlClass)

def traverseAst(cursor, inclusionConfig):

if (cursor.kind == clang.cindex.CursorKind.CLASS_DECL

or cursor.kind == clang.cindex.CursorKind.STRUCT_DECL

or cursor.kind == clang.cindex.CursorKind.CLASS_TEMPLATE):

# if the current cursor is a class, class template or struct declaration,

# we process it further ...

processClass(cursor, inclusionConfig)

for child_node in cursor.get_children():

traverseAst(child_node, inclusionConfig)

def parseTranslationUnit(filePath, includeDirs, inclusionConfig):

clangArgs = ['-x', 'c++'] + ['-I' + includeDir for includeDir in includeDirs]

tu = index.parse(filePath, args=clangArgs, options=clang.cindex.TranslationUnit.PARSE_SKIP_FUNCTION_BODIES)

for diagnostic in tu.diagnostics:

logging.debug(diagnostic)

logging.info('Translation unit:' + tu.spelling + "\n")

traverseAst(tu.cursor, inclusionConfig)

if __name__ == "__main__":

parser = argparse.ArgumentParser(description="CodeDependencyVisualizer (CDV)")

parser.add_argument('-d', required=True, help="directory with source files to parse (searches recusively)")

parser.add_argument('-o', '--outFile', default='uml.dot', help="output file name / name of generated dot file")

parser.add_argument('-u', '--withUnusedHeaders', help="parse unused header files (slow)")

parser.add_argument('-a', '--associations', action="store_true", help="draw class member assiciations")

parser.add_argument('-i', '--inheritances', action="store_true", help="draw class inheritances")

parser.add_argument('-p', '--privMembers', action="store_true", help="show private members")

parser.add_argument('-t', '--protMembers', action="store_true", help="show protected members")

parser.add_argument('-P', '--pubMembers', action="store_true", help="show public members")

parser.add_argument('-I', '--includeDirs', help="additional search path(s) for include files (seperated by space)", nargs='+')

parser.add_argument('-v', '--verbose', action="store_true", help="print verbose information for debugging purposes")

parser.add_argument('--excludeClasses', help="classes matching this pattern will be excluded")

parser.add_argument('--includeClasses', help="only classes matching this pattern will be included")

args = vars(parser.parse_args(sys.argv[1:]))

filesToParsePatterns = ['*.cpp', '*.cxx', '*.c', '*.cc']

if args['withUnusedHeaders']:

filesToParsePatterns += ['*.h', '*.hxx', '*.hpp']

filesToParse = findFilesInDir(args['d'], filesToParsePatterns)

subdirectories = [x[0] for x in os.walk(args['d'])]

loggingFormat = "%(levelname)s - %(module)s: %(message)s"

logging.basicConfig(format=loggingFormat, level=logging.INFO)

if args['verbose']:

logging.basicConfig(format=loggingFormat, level=logging.DEBUG)

logging.info("found " + str(len(filesToParse)) + " source files.")

for sourceFile in filesToParse:

logging.info("parsing file " + sourceFile)

parseTranslationUnit(sourceFile, args['includeDirs'], {

'excludeClasses': args['excludeClasses'],

'includeClasses': args['includeClasses']})

dotGenerator.setDrawAssociations(args['associations'])

dotGenerator.setDrawInheritances(args['inheritances'])

dotGenerator.setShowPrivMethods(args['privMembers'])

dotGenerator.setShowProtMethods(args['protMembers'])

dotGenerator.setShowPubMethods(args['pubMembers'])

dotfileName = args['outFile']

logging.info("generating dotfile " + dotfileName)

with open(dotfileName, 'w') as dotfile:

dotfile.write(dotGenerator.generate())

参考链接

GCC支持在代码中对头文件是否存在的判断(__has_include)

我们在实际编写代码的时候，经常需要判断当前编译环境是否存在我们需要的头文件，如果不存在，则使用其他头文件代替。

以前这个操作都是通过外部的configure文件生成Makefile的时候指定。

最近的GCC已经增加了__has_include这个内置函数判断头文件是否存在。

这个功能最早是Clang实现的，现在GCC终于补上了这个功能。

例子如下：

#ifdef __has_include
#  if __has_include(<optional>)
#    include <optional>
#    define have_optional 1
#  elif __has_include(<experimental/optional>)
#    include <experimental/optional>
#    define have_optional 1
#    define experimental_optional
#  else
#    define have_optional 0
#  endif
#endif

#ifdef __has_include

# if __has_include(<optional>)

# include <optional>

# define have_optional 1

# elif __has_include(<experimental/optional>)

# include <experimental/optional>

# define have_optional 1

# define experimental_optional

# else

# define have_optional 0

# endif

#endif

Code Composer Studio 8.2.0.00007 ,GCC 5支持这个内置函数。

参考链接

x86架构实现ARM架构下的SSAT指令

最近在使用Matlab仿真ARM下的Q15,Q31相关的操作，涉及到翻译CMSIS库中的部分函数翻译到x86下运行的情况。

一般会遇到两个比较特殊的宏，一个是__CLZ宏，另一个是 __SSAT宏，前者直接使用__buildin_clz替换就可以非常正常的工作，后者就比较复杂。

/**
  \brief   Signed Saturate
  \details Saturates a signed value.
  \param [in]  value  Value to be saturated
  \param [in]    sat  Bit position to saturate to (1..32)
  \return             Saturated value
 */
#define __SSAT                            __ssat

/**

\brief Signed Saturate

\details Saturates a signed value.

\param [in] value Value to be saturated

\param [in] sat Bit position to saturate to (1..32)

\return Saturated value

#define __SSAT __ssat

可以使用如下方式来翻译这个宏：

#if defined  (__GNUC__)
    static inline int __SSAT_GUN(int32_t VAL, int32_t BITPOS) {
      int32_t min = -(1<<(BITPOS-1));
      int32_t max = (1<<(BITPOS-1)) - 1;
      if (VAL < min)
        return min;
      else if (VAL > max)
        return max;
      else
        return VAL;
    }
    #define __SSAT(VAL, BITPOS) __SSAT_GUN(VAL,BITPOS)
#else
    #define __SSAT(VAL, BITPOS) \
        _ssatl(VAL , 0, BITPOS)
#endif

#if defined(__GNUC__)
    #define __CLZ __builtin_clz
#else
     #define __CLZ __clz
#endif

#if defined (__GNUC__)

static inline int __SSAT_GUN(int32_t VAL, int32_t BITPOS) {

int32_t min = -(1<<(BITPOS-1));

int32_t max = (1<<(BITPOS-1)) - 1;

if (VAL < min)

return min;

else if (VAL > max)

return max;

else

return VAL;

}

#define __SSAT(VAL, BITPOS) __SSAT_GUN(VAL,BITPOS)

#else

#define __SSAT(VAL, BITPOS) \

_ssatl(VAL , 0, BITPOS)

#endif

#if defined(__GNUC__)

#define __CLZ __builtin_clz

#else

#define __CLZ __clz

#endif

int在32/64位系统中的字节数

64-bit data models
Data model	short (integer)	int	long (integer)	long long	pointers, size_t	Sample operating systems
SILP64	64	64	64	64	64	Classic UNICOS^[41] (versus UNICOS/mp, etc.)
ILP64	16	64	64	64	64	HAL Computer Systems port of Solaris to the SPARC64
LLP64, IL32P64	16	32	32	64	64	Microsoft Windows (x86-64 and IA-64) using Visual C++; and MinGW
LP64, I32LP64	16	32	64	64	64	Most Unix and Unix-like systems, e.g., Solaris, Linux, BSD, macOS. Windows when using Cygwin; z/OS

参考链接

32位和64位系统区别及int字节数
 64-Bit Programming Models: Why LP64?
64-bit computing
What decides the sizeof an integer?

CMake如何编译CUDA(.cu)源文件

现在的项目，如果需要用到计算加速，Nvidia的CUDA往往是首选。那么如何在CMake中编译写好的CUDA源代码，可以参考如下。

首先使用FIND_PACKAGE找到已经安装的CUDA，此时需要配置的环境变量等，应该已经自动配置完成了

SET(CUDA_VERSION 8.0)
FIND_PACKAGE(CUDA ${CUDA_VERSION} REQUIRED)
STRING (TOLOWER ${CMAKE_SYSTEM_NAME} LOWER_SYSTEM_NAME)
SET(CUDA_TARGET_INCLUDE  ${CUDA_TOOLKIT_ROOT_DIR}-${CUDA_VERSION}/targets/${CMAKE_HOST_SYSTEM_PROCESSOR}-${LOWER_SYSTEM_NAME}/include)

SET(CUDA_VERSION 8.0)

FIND_PACKAGE(CUDA ${CUDA_VERSION} REQUIRED)

STRING (TOLOWER ${CMAKE_SYSTEM_NAME} LOWER_SYSTEM_NAME)

SET(CUDA_TARGET_INCLUDE ${CUDA_TOOLKIT_ROOT_DIR}-${CUDA_VERSION}/targets/${CMAKE_HOST_SYSTEM_PROCESSOR}-${LOWER_SYSTEM_NAME}/include)

接下来，使用CUDA_ADD_LIBRARY取代原来的ADD_LIBRARY，如下：

CUDA_ADD_LIBRARY(mylib SHARED
    file1.cpp
    file2.cu
    file3.cpp
    OPTIONS -arch sm_20
)

TARGET_LINK_LIBRARIES(mylib ${CUDA_LIBRARIES})

CUDA_ADD_LIBRARY(mylib SHARED

file1.cpp

file2.cu

file3.cpp

OPTIONS -arch sm_20

)

TARGET_LINK_LIBRARIES(mylib ${CUDA_LIBRARIES})

如果是可执行程序，请使用CUDA_ADD_EXECUTABLE取代ADD_EXECUTABLE。

参考链接

CMake: how to add cuda to existing project

在Makefile中检查gcc版本

问题描述

我想使用一些在旧的GCC版本中不可用的GCC警告开关(例如-Wtype-limits)。

有没有一个简单的方法来检查GCC版本？

最佳解决方案

我不会说容易，但是可以使用GNU make的SHELL函数来执行gcc --version命令，然后使用ifeq条件表达式来检查版本号，并适当地设置CFLAGS变量。

以下是一个简单的MakeFile示例：

CC = gcc
GCCVERSION = $(shell gcc --version | grep ^gcc | sed 's/^.* //g')
CFLAGS = -g

ifeq "$(GCCVERSION)" "4.4.3"
    CFLAGS += -Wtype-limits
endif

all:
        $(CC) $(CFLAGS) prog.c -o prog

CC = gcc

GCCVERSION = $(shell gcc --version | grep ^gcc | sed 's/^.* //g')

CFLAGS = -g

ifeq "$(GCCVERSION)" "4.4.3"

CFLAGS += -Wtype-limits

endif

all:

$(CC) $(CFLAGS) prog.c -o prog

注意：不存在ifgt语法。但是可以使用expr命令进行比较。

例子如下：

CC = gcc
GCCVERSIONGTEQ4 := $(shell expr `gcc -dumpversion | cut -f1 -d.` \>= 4)
CFLAGS = -g

ifeq "$(GCCVERSIONGTEQ4)" "1"
    CFLAGS += -Wtype-limits
endif

all:
        $(CC) $(CFLAGS) prog.c -o prog

CC = gcc

GCCVERSIONGTEQ4 := $(shell expr `gcc -dumpversion | cut -f1 -d.` \>= 4)

CFLAGS = -g

ifeq "$(GCCVERSIONGTEQ4)" "1"

CFLAGS += -Wtype-limits

endif

all:

$(CC) $(CFLAGS) prog.c -o prog

次佳解决方案

要将完整的3部分GCC版本(不仅第一位数字)转换成数字格式，适合比较(例如40701)使用

gcc -dumpversion | sed -e 's/\.\([0-9][0-9]\)/\1/g' -e 's/\.\([0-9]\)/0\1/g' -e 's/^[0-9]\{3,4\}$/&00/'

1	gcc -dumpversion \| sed -e 's/\.$[0-9][0-9]$/\1/g' -e 's/\.$[0-9]$/0\1/g' -e 's/^[0-9]\{3,4\}$/&00/'

其中解决了任何版本部分中双数字版本号的可能性，以及gcc -dumpversion输出内容缺少3个部分的可能性(在一些较早的GCC版本中存在这种情况)。

所以要测试MakeFile中的版本，使用(注意$$里面的最后一个sed命令)

GCC_GTEQ_472 := $(shell expr `gcc -dumpversion | sed -e 's/\.\([0-9][0-9]\)/\1/g' -e 's/\.\([0-9]\)/0\1/g' -e 's/^[0-9]\{3,4\}$$/&00/'` \>= 40702)
ifeq "$(GCC_GTEQ_472)" "1"
  ...
endif

GCC_GTEQ_472 := $(shell expr `gcc -dumpversion | sed -e 's/\.$[0-9][0-9]$/\1/g' -e 's/\.$[0-9]$/0\1/g' -e 's/^[0-9]\{3,4\}$$/&00/'` \>= 40702)

ifeq "$(GCC_GTEQ_472)" "1"

...

endif

第三种解决方案

我刚刚遇到这个问题，我需要测试GCC的前两位数字，想要一个比上面sed过滤字符串更复杂的操作。我使用bc进行比较，因为它支持浮点数(expr将非整数（non-integers）都视为字符串)：

GCC_VER_GTE44 := $(shell echo `gcc -dumpversion | cut -f1-2 -d.` \>= 4.4 | bc )
ifeq ($(GCC_VER_GTE44),1)
...
endif

GCC_VER_GTE44 := $(shell echo `gcc -dumpversion | cut -f1-2 -d.` \>= 4.4 | bc )

ifeq ($(GCC_VER_GTE44),1)

...

endif

如果在GCC 4.9之后发布了GCC 4.10，那么用sed处理是必要的：

GCC_VER_GTE44 := $(shell echo `gcc -dumpversion | cut -f1-2 -d.` \>= 4.4 | sed -e 's/\./*100+/g' | bc )
ifeq ($(GCC_VER_GTE44),1)
...
endif

GCC_VER_GTE44 := $(shell echo `gcc -dumpversion | cut -f1-2 -d.` \>= 4.4 | sed -e 's/\./*100+/g' | bc )

ifeq ($(GCC_VER_GTE44),1)

...

endif

参考链接

在Makefile中检查gcc版本？

Ubuntu 16.04系统上Clang与GCC之间切换

在编译C++代码的时候，我们有时需要比较一下不同编译器之间优化性能的差异，因此需要在Clang与GCC之间进行切换，用来比较最后的实际效果。

在Ubuntu 16.04系统上使用如下命令进行切换

$ sudo apt-get install clang
$ sudo update-alternatives --config c++

1 2	$ sudo apt-get install clang $ sudo update-alternatives --config c++

参考链接

Switching between GCC and Clang/LLVM using CMake

Ubuntu 16.04上Linux C++程序性能分析工具perf使用入门

目前，perf是Linux系统上最全面最方便的一个性能检测工具。由Linux内核携带并且同步更新。

Ubuntu 16.04系统上需要执行如下命令安装：

# 常规内核使用
#sudo apt-get install linux-tools-generic
# 低延时内核使用
#sudo apt-get install linux-tools-lowlatency

# 因此我们根据内核自动选择安装
$ sudo apt-get install linux-tools-`uname -r | cut -d- -f1-2`-`uname -r | cut -d- -f3`

$ sudo apt-get install linux-tools-common

#解决报错"Kernel address maps (/proc/{kallsyms,modules}) were restricted. Check /proc/sys/kernel/kptr_restrict before running 'perf record'".

$ sudo sh -c " echo 0 > /proc/sys/kernel/kptr_restrict"

# 常规内核使用

#sudo apt-get install linux-tools-generic

# 低延时内核使用

#sudo apt-get install linux-tools-lowlatency

# 因此我们根据内核自动选择安装

$ sudo apt-get install linux-tools-`uname -r | cut -d- -f1-2`-`uname -r | cut -d- -f3`

$ sudo apt-get install linux-tools-common

#解决报错"Kernel address maps (/proc/{kallsyms,modules}) were restricted. Check /proc/sys/kernel/kptr_restrict before running 'perf record'".

$ sudo sh -c " echo 0 > /proc/sys/kernel/kptr_restrict"

使用方法如下（gcc编译时最好使用-g参数，生成符号，方便调试）：

#生成性能日志文件,默认生成 perf.data
$ sudo perf record -e cpu-clock -g ./hello

#解析性能日志
$ perf report -g -i perf.data

#生成性能日志文件,默认生成 perf.data

$ sudo perf record -e cpu-clock -g ./hello

#解析性能日志

$ perf report -g -i perf.data

参考链接

软件的性能是软件质量的重要考察点，不论是在线服务程序还是离线程序，甚至是终端应用，性能都是用户体验的关键。这里说的性能重大的范畴来讲包括了性能和稳定性两个方面，我们在做软件测试的时候也是要重点测试版本的性能表现和稳定性的。对于软件测试过程中发现的性能问题，如何定位有很多的方法。基本的方法可能是开发者对代码进行review，或者是使用一些工具对代码进行性能分析。常见的性能分析工具有哪些呢？以下两篇文章做了详细的总结：

gprof是可用于Linux C++代码性能分析工具之一。
继续阅读Linux C++程序进行性能分析工具gprof使用入门

2021年4月
一	二	三	四	五	六	日
			1	2	3	4
5	6	7	8	9	10	11
12	13	14	15	16	17	18
19	20	21	22	23	24	25
26	27	28	29	30