Developer Guide
Here resides the documentation for the reducto modules. This may be interesting if you are curious about how reducto works under the hoods.
cli
Contains a single function which would be generated when running flit install locally, or when pip installing the package.
Call the Command Line Interface here.
reducto
Contains the class representing the application. When the program is invoked, the code called from main is defined in this module.
Module containing the application abstraction.
- class reducto.reducto.Reducto[source]
Class defining the package application.
This class represents the reducto application.
Abstracts the application to be used as a command line interface by means of argparse.ArgumentParser.
The different arguments for the app are defined as private methods of this class.
reports
This module defines the reporting facilities defined. There are two classes, SourceReport for single files, and PackageReport to deal with a whole package.
Module storing the different reports presented by the package.
- class reducto.reports.ReportFormat(value)[source]
Formats allowed for the reports.
JSON corresponds to the base dict format, the remaining formats correspond to the ones defined in tabulate package.
- exception reducto.reports.ReportFormatError(fmt: reducto.reports.ReportFormat)[source]
Error raised on wrong reporting format.
- class reducto.reports.SourceReport(src_file: SourceFile)[source]
Reporting class per a source (.py) file.
Contains a report method to obtain the proper report format from a SourceFile object.
- report()[source]
See also
SourceFile- report(fmt: reducto.reports.ReportFormat = ReportFormat.JSON, is_package: bool = False, percentage: bool = False) Union[str, Dict[str, Dict[str, Any]]][source]
Report of a source file.
- Parameters
fmt (ReportFormat) – Must be one of ReportFormats. Defaults to ReportFormats.JSON.
is_package (bool) – Bool to determine if a SourceFile is the single entry point for an app. Defaults to False.
percentage (bool) – Whether to report the lines as percentage or not. Defaults to False
- Returns
report
- Return type
ReportDict
- Raises
ReportFormatError – When the reporting required is not defined in ReportFormat enum.
- property source_file: SourceFile
Returns the source file.
- Returns
src_file
- Return type
src.SourceFile.
- class reducto.reports.PackageReport(package: Package)[source]
Define report for a package, gets a pkg.Package as input.
Contains a report method to obtain the proper report format from a SourceFile object.
See also
Package- property package: Package
Returns the package given as input.
- Returns
package
- Return type
Package
- report(fmt: reducto.reports.ReportFormat = ReportFormat.JSON, grouped: bool = False, percentage: bool = False) Union[str, Dict[str, Any], Dict[str, Dict[str, Any]]][source]
Report method for a package.
Generates the report for a Package made of Source files. Initially gets the info either grouped or ungrouped, if the format chosen is json its returned directly.
- Parameters
fmt (ReportFormat) – Format to return the information. Defaults to ReportFormats.JSON.
grouped (bool) – Whether to return the information by source files, or grouped at the package level (resumes the package). Defaults to False, returns the information per source file.
percentage (bool) – Whether to report the lines as percentage or not. Defaults to False
- Returns
report
- Return type
ReportPackageDict
- Raises
ReportFormatError – When a report format is not defined
package
Contains the code which deals with a python package traversal. Package class parses a directory tree to obtain the python source files, instantiating the corresponding *SourceFile*s.
Module containing code to control python package crawling.
- exception reducto.package.PackageError[source]
Error raised when a directory is not a valid python package.
- class reducto.package.Package(path: pathlib.Path)[source]
Class controlling a package.
Entry point for reducto. Controls what a python package is and generates the associated source files to be parsed. The general content of a package is grabbed from here.
- property average_function_length: int
Average function length for the package.
Only the source_lines are taken into account.
- Returns
average – Average function length, rounded to the closest int.
- Return type
- property average_function_lengths: List[int]
Average function lengths per surce file.
- Returns
averages – Average length of the functions per source file.
- Return type
List[int]
- property blank_lines: List[int]
Returns a list with the number of blank lines per source file.
- Returns
blank_lines – Number of blank lines per source_file.
- Return type
List[int]
- property comment_lines: List[int]
Returns a list with the number of comment lines per source file.
- Returns
comment_lines – Number of comment lines per source_file.
- Return type
List[int]
- property docstrings: List[int]
Returns a list with the number of docstring lines per source file.
- Returns
docstrings – Number of docstrings per source_file.
- Return type
List[int]
- property functions: List[List[reducto.items.FunctionDef]]
Returns a list of lists of FunctionDef objects.
- Returns
functions – FunctionDef obtained in each source_file.
- Return type
List[List[it.FunctionDef]]
See also
reducto.items.FunctionDef
- property lines: List[int]
Returns the list of lines of each file.
- Returns
lines – Number of lines of each of the source_files.
- Return type
List[int]
- property number_of_functions: List[int]
Returns a list of of number of functions per source_file.
- Returns
number_of_functions – Number of FunctionDef per source_file.
- Return type
List[int]
- property path: pathlib.Path
Returns the full path pointing to the package.
- Returns
path
- Return type
Path
- report() reducto.reports.PackageReport[source]
Obtains the PackageReport.
- Returns
reporter – Reporter object to obtain a proper data structure to present the results.
- Return type
rp.PackageReport.
See also
- property source_files: List[reducto.src.SourceFile]
Returns the list of source files.
- Returns
source_files
- Return type
List[src.SourceFile]
- property source_lines: List[int]
Returns a list with the number of source code lines per source file.
- Returns
source_lines – Number of source lines of each source file.
- Return type
List[int]
- static validate(path: pathlib.Path) None[source]
Check if the input path is a proper python package.
If the directory is either a package or an src package, returns None, otherwise raises PackageError.
- Parameters
path (Path) –
- Raises
PackageError – When the directory is not a valid python package.
- reducto.package.is_package(path: pathlib.Path) bool[source]
Checks whether a given folder is a python package or not.
├─ packagename │ ├─ __init__.py │ └─ …
- Parameters
path (Path) –
- Returns
check – Returns True if a path is a directory and contains an __init__.py file inside, False otherwise.
- Return type
- reducto.package.is_src_package(path: pathlib.Path) bool[source]
Checks whether a package is of the form:
├─ src │ └─ packagename │ ├─ __init__.py │ └─ … ├─ tests │ └─ … └─ setup.py
The check for the path will be if its a directory with only one subdirectory containing an __init__.py file.
- Parameters
path (Path) – Full path pointing to a dir.
- Returns
check – If the package is an src package, returns True, False otherwise.
- Return type
See also
src
This module contains the definition of a python source file represented by the SourceFile class.
The source files are parsed using the ast to obtain the information regarding the functions (and methods inside classes, treated as functions), as well as docstrings, and the tokenize module to obtain the information related to comments and blank lines.
Module containing the functionalities to parse a python source file.
Uses ast and tokenize builtin libraries.
References
- class reducto.src.SourceFile(filename: pathlib.Path)[source]
Class representing a .py source file.
Allows to read a file, obtain the tokens and the ast.
- property ast: _ast.Module
Parses and returns the ast of a file from the lines read with tokenize module.
- Returns
ast – Abstract Syntax Tree module object.
- Return type
ast.Module
- property blank_lines: int
Returns the total number of lines which are blank lines.
- Returns
blank_lines
- Return type
See also
- property blank_lines_positions: List[int]
Returns the list of positions of the blank lines.
This is used to register the functions.
- Returns
blank_lines_positions – List with the positions of the blank lines.
- Return type
List[int]
- property comment_lines: int
Returns the total number of lines which are comments.
- Returns
comments
- Return type
See also
- property comment_lines_positions: List[int]
Returns the list of positions of the comment lines.
This is used to register the functions.
- Returns
comments_positions – List with the line position of each comment line.
- Return type
List[int]
- property functions: List[reducto.items.FunctionDef]
Returns the list of functions grabbed through the ast.
- Returns
functions – List of FunctionDef objects in the file.
- Return type
List[it.FunctionDef]
See also
reducto.items.FunctionDef
- property lines: List[str]
Contains the lines of the file as initially parsed with tokenize module.
- property module_docstrings: int
Obtains the lines of docstrings at the module level.
- Returns
docstrings – Lines of docstrings in the module.
- Return type
- report() reducto.reports.SourceReport[source]
Obtain the reporter class.
- Returns
reporter
- Return type
rp.SourceReport.
See also
reducto.reports.SourceReport
- property source_lines: int
Computes the total number of source code lines of the file.
- Returns
source_lines – Source lines are computed as the total number of lines minus the docstrings, minus the comment lines and minus the blank lines.
- Return type
- property source_visitor: reducto.src.SourceVisitor
Returns the SourceVisitor once visited.
Instantiates and calls the visit method of the SourceVisitor. All the necessary info regarding the ast of the source file should be contained here.
- Returns
source_visitor
- Return type
SourceVisitor
See also
SourceVisitor
- property tokens: List[tokenize.TokenInfo]
Return the complete set of tokens for a file.
See also
- property total_docstrings: int
Get the total number of docstring lines.
The total number of docstrings are the module level docstrings plus the ones recorded from the functions.
- Returns
total_docstrings
- Return type
- static validate(path: pathlib.Path) None[source]
Check if the path is a valid python file.
Called on instantiation. It must be a file with the extension .py.
- Parameters
path (Path) – Path to validate.
- Returns
check – True if is a valid python file, false otherwise.
- Return type
- Raises
SourceFileError – When the file is not a valid python file.
- reducto.src.token_is_comment_line(tok: tokenize.TokenInfo) bool[source]
Checks whether a given token line is a comment.
The check is done for the whole line, not just the token type. A line containing a comment after anything which isn’t a space is not considered a comment line.
- Parameters
tok (tokenize.TokenInfo) – Token as extracted from tokenize.generate_tokens.
- Returns
is_comment_line – Returns True when the line is a comment line only, False otherwise.
- Return type
- reducto.src.token_is_blank_line(tok: tokenize.TokenInfo) bool[source]
Checks if a given line is a blank line or not.
The check is done for a NL token and a line containing only a new line character.
- Parameters
tok (tokenize.TokenInfo) – Token obtained from tokenize.generate_tokens.
- Returns
check – If a token is a blank line returns True, False otherwise.
- Return type
Examples
>>> tok = tokenize.TokenInfo(type=61, string='\n', start=(123, 0), end=(126, 1), line='\n') >>> token_is_blank_line(tok) True
- class reducto.src.SourceVisitor[source]
Bases:
ast.NodeVisitorClass inheriting from ast.NodeVisitor.
Defines the actions to be taken on the ast being parsed. Obtains every FunctionDef in the source file, (included async ones) and when a class is found, the corresponding methods are parsed just like FunctionDef objects. No other object is obtained from the SourceVisitor. Instantiates the corresponding items and registers the content, like the number of lines and docstrings.
TODO: - Añadir la distinción entre método y función (método nuevo en lugar de generic_visit?)
Notes
For the moment, no distinction is done between functions and definitions, so between the items there will be only FunctionDef, no MethodDef items.
See also
- property functions: List[reducto.items.FunctionDef]
Returns the items which are functions from the list of items obtained.
When the functions are retrieved, the docstrings are obtained registered.
- Returns
functions
- Return type
List[it.FunctionDef]
- property items: List[reducto.items.Item]
List of items parsed with the ast.parse.
- Returns
items – Returns all the items found in the ast source.
- Return type
List[it.Item]
See also
reducto.items.Item
- register_functions(content: Dict[str, List[int]]) None[source]
Register the contents grabbed as tokens outside the ast.
Inserts the positions of the comments and blank lines.
- Parameters
content (Dict[str, List[int]]) – Contains a dict with the list of the corresponding positions to be registered.
Examples
>>> content = {'comments': [1, 4, 5, 6, 22], 'blank_lines': [12, 46]} >>> visitor.register_functions(content)
- visit_AsyncFunctionDef(node)
Visits the FunctionDef nodes.
Creates and stores the corresponding item, along with the name, start and end lines.
- Parameters
node (ast.FunctionDef) – Node automatically filtered when visit method is called.
- Returns
node – Returns the node itself.
- Return type
See also
reducto.items.FunctionDefNotes
Typing the argument raises mypy error.
- visit_ClassDef(node: _ast.ClassDef)[source]
Calls the general generic_visit method on itself, obtaining the FunctionDef objects found.
- visit_FunctionDef(node)[source]
Visits the FunctionDef nodes.
Creates and stores the corresponding item, along with the name, start and end lines.
- Parameters
node (ast.FunctionDef) – Node automatically filtered when visit method is called.
- Returns
node – Returns the node itself.
- Return type
See also
reducto.items.FunctionDefNotes
Typing the argument raises mypy error.
items
When a source file is parsed, the relevant elements obtained from the ast are represented as items. The only parsed ast node are functions and methods, both represented currently by FunctionDef. These items will carry the information to compute the source lines, docstrings, etc.
Contains the elements to be extracted from a source file.
- class reducto.items.Item(name: str, start: int = 0, end: int = 0)[source]
Base class for the items to be extracted from an ast parsed source file.
A subclass of this Item corresponds to an ast Node.
Notes
The nodes are inserted as an attribute instead of doing it on the initialization to simplify the unit tests.
- property blank_lines: int
Number of lines which are blank lines in the item.
- Returns
blank_lines
- Return type
- property comments: int
Number of lines which are comments in the item.
- Returns
comments
- Return type
- property docstrings: int
Number of lines which are docstring in the item.
- Returns
docstrings
- Return type
- property node: _ast.AST
Returns the node itself.
- Returns
node
- Return type
- property source_lines: int
Computes the total number of lines of the item.
- Returns
source_lines – The total number of lines is the len of the function minus docstrings, comment lines and blank lines.
- Return type
- class reducto.items.FunctionDef(name: str, start: int = 0, end: int = 0)[source]
Bases:
reducto.items.ItemImplementation of an ast.FunctionDef.
No distinction to an AsyncFunctionDef is made.
- class reducto.items.MethodDef(name: str, start: int = 0, end: int = 0)[source]
Bases:
reducto.items.FunctionDefEquivalent to a FunctionDef.
Defined in case a distinction between functions and methods is implemented. NOT USED.
- reducto.items.get_docstring_lines(node: Union[_ast.Module, _ast.FunctionDef, _ast.AST]) int[source]
Obtains the number of lines which are docstrings.
Uses ast.get_docstring to extract the docstrings of an ast node.
- Parameters
node (Union[ast.Module, ast.FunctionDef]) – Node which may be a container of docstrings. Only defined for Module and FunctionDed.
- Returns
docstring_lines – Number of lines in the function or module which are docstrings.
- Return type
Notes
Reminder: - ‘’’first ‘’’ 1 - ‘’’second ‘’’ 2 - ‘’’
third’’’ 1 - ‘’’ fourth ‘’’ 1 - ‘’’docs
‘’’ 4