DFTracer: is a library for profiling I/O calls and application functions.

User Guide

• Overview
• Definitions
• Limitations
  • Spawning of processes
  • OS Compatibility
  • Write Buffering
• Build DFTracer
  • Build DFTracer with pip (Recommended)
  • Build DFTracer with Spack
  • Build DFTracer with CMake
  • Build Variables
• Building applications with DFTracer Code Annotations
  • Building for Projects with Makefiles
• DFTracer APIs
  • Modes of DFTracer
  • Configurations of DFTracer
  • DFTracer C++ APIs
  • DFTracer C APIs
  • DFTracer C/C++ Function Profiling using GCC
  • DFTracer Python APIs
• DFTracer Format
  • Complete Events
  • Metadata Events

DFAnalyzer

• Build
  • From source (Recommended)
  • From pip
• Getting Started with DFAnalyzer
• Running DFAnalyzer with Dask Distributed
  • Getting Started
  • Starting a Dask Distributed Cluster
  • Use DFAnalyzer
  • Acessing the Dask Dashboard
  • Stopping Dask Distributed Workers
• Running Dask Distributed in a new system
  • Initializing Dask Configurations
  • Editing $HOME/.dftracer/configuration.yaml
  • Editing Dask Configurations depending on your System
  • Run DFAnalyzer
• ALCF Polaris
  • Logging in to Polaris Login Node
  • Initializing Dask Configurations
  • Changing $HOME/.dftracer/configuration.yaml
  • Changing polaris.yaml config
  • Executing scheduler
  • Forwarding the Port
  • Opening Notebook File
  • Stopping Dask Distributed Workers
  • Tips and Tricks
• Overlap Analysis
  • Overlap vs Non-overlap Analysis
  • Annotating using DFTracer
  • Analysis using DFAnalyzer

Utilities

• DFTracer Utility scripts
  • Merge Trace script
  • Compaction script
  • Sanitize script
  • Anonymize script
  • Create Index script
  • Splitting DFTracer traces
  • Counting DFTracer traces’ events
  • Validating DFTracer traces
• Bash Utility scripts
  • Handling .pfw files
  • Handling .pfw.gz files
  • Extracting JSON data
• Perfetto UI
  • Loading traces
  • Using SQL Query

Reference

• C++ Example
  • Application Level Example:
  • LD_PRELOAD Example:
  • Hybrid Example:
• C Example
  • Application Level Example:
  • LD_PRELOAD Example:
  • Hybrid Example:
• Python Example
  • Application Level Example:
  • LD_PRELOAD Example:
  • Hybrid Example:
• Resnet50 with pytorch and torchvision example from ALCF Polaris:
• Integrated Applications
• Example Chrome Tracing Plots
• Migration from DLIO Profiler to DFTracer
  • Installation
  • Application building migration
  • C++ API Changes
  • C API Changes
  • Python API changes
  • Application building migration
  • C++ API Changes
  • C API Changes
  • Python API changes
  • Analyzer Changes

Applications

• Pegasus Montage with DFTracer
  • Step 1: Install Condor
  • Step 2: Install Pegasus
  • Step 3: Install and compile Montage
  • Step 4: Get the montage-pegasus-v3
  • Step 5: Compile the pegasus-mpi-cluster from source
  • Step 6: Create a single “install” directory for all Pegasus software
  • Step 7: Prepare software for Pegasus-Montage
  • Step 8: Annotate Montage
• Pegasus 1000-Genome with DFTracer
  • Step 1: Activate Environment
  • Step 2: Get the 1000genome-workflow
  • Step 3: Prepare software for Pegasus-1000Genome
  • Step 4: Annotate 1000-Genome

Contributing

• Developer Guide
  • ALCF Polaris
• Style Guides
  • Coding Conventions
  • Commit Message Format
• Testing Guide
  • Unit Tests

Indices and tables

• Index

• Module Index

• Search Page