Debug a Custom Extractor

Add a main block in tap.py of your Custom Extractor #

if __name__ == "__main__":
    # TapCustomExtractor is the class name of your tap in tap.py
    TapCustomExtractor.cli()

Create a Local Venv That Your Debugger Can Use #

If you’re using Poetry you need to create a venv in your custom tap directory so that your IDE can pick it up when debugging.

If you have already installed poetry this will involve removing the existing poetry virtual environment. Running the command below and finally reinstalling poetry.

poetry config virtualenvs.in-project true

What to put in VSCode launch.json #

Add the following launch configuration to your project:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            # Replace tap_foobar below with the actual name of your custom extractors library
            "program": "${workspaceRoot}/tap_foobar/tap.py",
            "console": "integratedTerminal",
            "args": ["--config", ".secrets/config.json"],
            "env": { "PYTHONPATH": "${workspaceRoot}"},
            # Change this to false if you wish to debug and add breakpoints outside of your code e.g. the singer-sdk package
            "justMyCode": true

        }
    ]
}

Create a config.json to use when debugging #

The above launch.json specifies the location of this config as .secrets/config.json.

Feel free to change this but ensure the config has all the required config fields for your custom extractor to run successfully.

Happy Debugging! #

You should now be able to add breakpoints where you need and run the debugger.

References #

• Stack Overflow: How to debug a singer tap using vs code