OWASP - OWTF in a word¶

The OWASP - OWTF project provides an efficient approach to combine the power of automation with the out-of-the-box thinking that only the user can provide.

It gathers a complete set of plugins and merges their results into an interactive report. The user has then the possibility to add notes, to change details and to add media like screenshots in order to have a complete report.

The goals aimed by ptp¶

The primary goal of ptp is to enhance OWASP - OWTF in order to provide an automated ranking for each plugin. This will allow the user to focus attention on the most likely weak areas of a web application or network first, which will be valuable to efficiently use the remaining time in a penetration assessment.

Instead of evaluating every plugins run by OWASP - OWTF and defining the rankings for each of them, thanks to ptp, the user will be able to focus on the ones that have been ranked with the highest risks. The user is then able to confirm or override the automated rankings since we estimate that she/he is the only one that can accurately detect the false positives.

When developing the automated ranking system, ptp‘s main goal was joined with a secondary one. Apart from its main feature which is ranking the results from security tools reports, it also provides an unified way to reuse these reports directly in your python code, without having to deal with complex parsing.

Using pip¶

The ptp library is available on PyPI at the following address: https://pypi.python.org/pypi/ptp.

The easiest way to install it is using pip.

$ pip install ptp

From scratch¶

It is also possible to install the library from its repository. You will then be able to use the latest possible version or even try the develop branch.

The first step is to clone the repository of the project:

$ git clone https://github.com/owtf/ptp.git

Then run the setup.py script:

$ ./setup.py install

Auto-detection mode¶

The ptp module provides the ptp.PTP class that exposes the public API of the library.

The simplest way to use ptp.PTP is with the auto-detection mode. This mode tries to reduce as much as possible our work by auto-detecting which tool has generated a given report and use the corresponding libptp.parser.AbstractParser.

That way, we do not need to know if the report we want to parse has been generated by W3AF, DirBuster or even Skipfish.

Example:

>>> from ptp import PTP
>>> myptp = PTP(pathname='my/directory', filename='my_report')
>>> myptp.parse()
[{'ranking': 4}, ..., {'ranking': 3}, ..., {'ranking': 1}]

>>> from ptp import PTP
>>> myptp = PTP(pathname='my/directory')
>>> myptp.parse()
[{'ranking': 4}, ..., {'ranking': 3}, ..., {'ranking': 1}]

If we are only looking for the highest risk that is listed in the report, we can use the following function:

>>> myptp.get_highest_ranking()
4
>>> from libptp.constants import HIGH
>>> myptp.get_highest_ranking() == HIGH
True

Explicit mode¶

If we already know which tool has generated the report, we can explicitly give that information to ptp.PTP. That will even speed up the whole process since it will not have to lookup for the right parser.

The list of the supported tools can be found like below:

>>> PTP.supported
{
    'arachni': [<class 'libptp.tools.arachni.parser.ArachniXMLParser'>],
    'dirbuster': [<class 'libptp.tools.dirbuster.parser.DirbusterParser'>],
    'metasploit': [<class 'libptp.tools.metasploit.parser.MetasploitParser'>],
    'nmap': [<class 'libptp.tools.nmap.parser.NmapXMLParser'>],
        <class 'libptp.tools.wapiti.parser.WapitiXMLParser'>,
        <class 'libptp.tools.wapiti.parser.Wapiti221XMLParser'>
    ],
    'owasp-cm-008': [<class 'libptp.tools.owasp.cm008.parser.OWASPCM008Parser'>],
    'robots': [<class 'libptp.tools.robots.parser.RobotsParser'>]
    'skipfish': [<class 'libptp.tools.skipfish.parser.SkipfishJSParser'>],
    'wapiti': [
    'w3af': [<class 'libptp.tools.w3af.parser.W3AFXMLParser'>],
}

Example:

>>> myptp = PTP('skipfish')
>>> myptp.parse(pathname='my/other/directory')
[{'ranking': 2}, {'ranking': 2}, {'ranking': 1}]

Attributes¶

If we are interested in the name of the tool that generated the report, it is stored in the ptp.PTP.tool_name attribute and can be retrieved like below:

>>> print(myptp.tool_name)
arachni  # In our case, it is Arachni that has generated our report.

We can also retrieve the list of the vulnerabilities thanks to the ptp.PTP.vulns attribute:

>>> myptp.vulns
[{'ranking': 4}, ..., {'ranking': 3}, ..., {'ranking': 1}]

And the metadata thanks to the ptp.PTP.metadata attribute.

>>> myptp.metadata
{'version': 'a.b'}

The skeleton¶

By convention, the class name must contain the format it parses (in our case XML).

from libptp.parser import XMLParser


class MyXMLParser(XMLParser):
    """Specialized parser for MyTool."""

    __tool__ = 'mytool'
    __version__ = ['0.1']

    def __init__(self, pathname, filename='*.xml'):
        """Initialize MyXMLParser.

        :param str pathname: Path to the report directory.
        :param str filename: Regex matching the report file.

        """
        XMLParser.__init__(self, pathname, filename)

We added a couple of class attributes in order to give some information about what tool is parsed by our class and the supported versions.

Since our parser inherits from XMLParser, we do not have to specify the __format__ class attribute, which is already set to xml.

Matching the supported reports¶

The next step is to write the is_mine() class method which tells ptp whether or not it can parse the report file.

Let us say that MyTool‘s XML report has <mytool version=’x.x’> as the root XML tag.

Therefore, our is_mine() function is:

class MyXMLParser(XMLParser):
    """Specialized parser for MyTool."""

    __tool__ = 'mytool'
    __version__ = ['0.1']

    # Omitted unchanged code

    @classmethod
    def is_mine(cls, pathname, filename='*.xml'):
        """Check if it is a supported MyTool report.

        :param str pathname: Path to the report directory.
        :param str filename: Regex matching the report file.

        :return: `True` if it supports the report, `False` otherwise.
        :rtype: :class:`bool`

        """
        try:
            stream = cls.handle_file(pathname, filename)
        except (ValueError, LxmlError):
            # If an error occurs when trying to open the file, then the
            # parser cannot deal with it.
            return False
        # The root tag must contain 'mytool'.
        if not cls.__tool__ in stream.tag:
            return False
        # Check if the root node has a 'version' attribute.
        if not 'version' in stream:
            return False
        # Check if the version is the one this parser supports.
        if not stream.get('version') in cls.__version__:
            return False
        return True

Parsing methods¶

Each AbstractParser class has to provide two methods:

• libptp.parser.AbstractParser.parse_metadata() which parses the metadata of the report and formats them into a dict.
• libptp.parser.AbstractParser.parse_report() which parses the discoveries that are listed in the report and formats them into a list of dict.

In order to keep it simple, we will not detail the implementations of these methods for our fake tool.

from libptp.parser import XMLParser


class MyXMLParser(XMLParser):
    """Specialized parser for MyTool."""

    __tool__ = 'mytool'
    __version__ = ['0.1']

    def __init__(self, pathname, filename='*.xml'):
        """Initialize MyXMLParser.

        :param str pathname: Path to the report directory.
        :param str filename: Regex matching the report file.

        """
        XMLParser.__init__(self, pathname, filename)

    @classmethod
    def is_mine(cls, pathname, filename='*.xml'):
        """Check if it is a supported MyTool report.

        :param str pathname: Path to the report directory.
        :param str filename: Regex matching the report file.

        :return: `True` if it supports the report, `False` otherwise.
        :rtype: :class:`bool`

        """
        try:
            stream = cls.handle_file(pathname, filename)
        except (ValueError, LxmlError):
            # If an error occurs when trying to open the file, then the
            # parser cannot deal with it.
            return False
        # The root tag must contain 'mytool'.
        if not cls.__tool__ in stream.tag:
            return False
        # Check if the root node has a 'version' attribute.
        if not 'version' in stream:
            return False
        # Check if the version is the one this parser supports.
        if not stream.get('version') in cls.__version__:
            return False
        return True

    def parse_metadata(self):
        return {}  # The expected behavior is to return a dict.

    def parse_report(self):
        return []  # The expected behavior is to return a list.

AbstractParser¶

XMLParser¶

FileParser¶

LineParser¶

Parser¶

Parser¶

Signatures¶

Parser¶

Signatures¶

Parser¶

CM-008¶

OWASP-CM-008 tests the HTTP methods of a website that are available.

Parser¶

Signatures¶

Parser¶

Parser¶

Parser¶

Signatures¶