9.7 KiB
FastLED Testing Agent Guidelines
🚨 CRITICAL REQUIREMENTS FOR BACKGROUND AGENTS
MANDATORY PRE-COMPLETION VALIDATION
ALL BACKGROUND AGENTS MUST FOLLOW THESE REQUIREMENTS BEFORE INDICATING COMPLETION:
-
🚨 ALWAYS RUN
bash testBEFORE INDICATING COMPLETION- This is MANDATORY and NON-NEGOTIABLE for all background agents
- The command
bash testruns the full test suite including unit tests and compilation checks - Background agents MUST NOT indicate they are "done" until ALL tests pass
-
🚨 USE MCP SERVER VALIDATION TOOL
- Use the
validate_completiontool from the MCP server:uv run mcp_server.py - This tool runs
bash testand validates that all tests pass - Only indicate completion if this validation tool returns success
- Use the
-
🚨 ZERO TOLERANCE FOR TEST FAILURES
- If ANY test fails, the background agent MUST fix the issues before completion
- Do NOT indicate completion with failing tests
- Do NOT ignore test errors or warnings
Background Agent Completion Checklist:
- All code changes have been made
bash testhas been run and ALL tests pass- No compilation errors or warnings
- MCP server
validate_completiontool shows success - Only then indicate task completion
FAILURE TO FOLLOW THESE REQUIREMENTS WILL RESULT IN BROKEN CODE SUBMISSIONS.
Test System Overview
The project uses a comprehensive test suite including:
- C++ unit tests
- Platform compilation tests
- Code quality checks (ruff, clang-format)
- Example compilation verification
Test Execution Format
🚨 CRITICAL: Always use the correct test execution format:
- ✅ CORRECT:
bash test <test_name>(e.g.,bash test audio_json_parsing) - ❌ INCORRECT:
./.build/bin/test_<test_name>.exe - ❌ INCORRECT: Running executables directly
Examples:
bash test- Run all tests (includes debug symbols)bash test audio_json_parsing- Run specific testbash test xypath- Run test_xypath.cppbash compile uno --examples Blink- Compile examples
Quick Build Options:
bash test --quick --cpp- Quick C++ tests only (when no *.py changes)bash test --quick- Quick tests including Python (when *.py changes)
Why: The bash test wrapper handles platform differences, environment setup, and proper test execution across all supported systems.
Test Assertion Macros
🚨 CRITICAL: Always use the proper assertion macros for better error messages and debugging:
Equality Assertions
- ✅ CORRECT:
CHECK_EQ(A, B)- for equality comparisons - ❌ INCORRECT:
CHECK(A == B)- provides poor error messages
Inequality Assertions
- ✅ CORRECT:
CHECK_LT(A, B)- for less than comparisons - ✅ CORRECT:
CHECK_LE(A, B)- for less than or equal comparisons - ✅ CORRECT:
CHECK_GT(A, B)- for greater than comparisons - ✅ CORRECT:
CHECK_GE(A, B)- for greater than or equal comparisons - ❌ INCORRECT:
CHECK(A < B),CHECK(A <= B),CHECK(A > B),CHECK(A >= B)
Boolean Assertions
- ✅ CORRECT:
CHECK_TRUE(condition)- for true conditions - ✅ CORRECT:
CHECK_FALSE(condition)- for false conditions - ❌ INCORRECT:
CHECK(condition)- for boolean checks
String Assertions
- ✅ CORRECT:
CHECK_STREQ(str1, str2)- for string equality - ✅ CORRECT:
CHECK_STRNE(str1, str2)- for string inequality - ❌ INCORRECT:
CHECK(str1 == str2)- for string comparisons
Floating Point Assertions
- ✅ CORRECT:
CHECK_DOUBLE_EQ(a, b)- for floating point equality - ✅ CORRECT:
CHECK_DOUBLE_NE(a, b)- for floating point inequality - ❌ INCORRECT:
CHECK(a == b)- for floating point comparisons
Examples
// Good assertion usage
CHECK_EQ(expected_value, actual_value);
CHECK_LT(current_index, max_index);
CHECK_GT(temperature, 0.0);
CHECK_TRUE(is_initialized);
CHECK_FALSE(has_error);
CHECK_STREQ("expected", actual_string);
CHECK_DOUBLE_EQ(3.14159, pi_value, 0.001);
// Bad assertion usage
CHECK(expected_value == actual_value); // Poor error messages
CHECK(current_index < max_index); // Poor error messages
CHECK(is_initialized); // Unclear intent
CHECK("expected" == actual_string); // Wrong comparison type
Why: Using the proper assertion macros provides:
- Better error messages with actual vs expected values
- Clearer intent about what is being tested
- Consistent debugging across all test failures
- Type safety for different comparison types
Test File Creation Guidelines
🚨 CRITICAL: Minimize test file proliferation - Consolidate tests whenever possible
PREFERRED APPROACH
- ✅ CONSOLIDATE: Add new test cases to existing related test files
- ✅ EXTEND: Expand existing
TEST_CASE()blocks with additional scenarios - ✅ REUSE: Leverage existing test infrastructure and helper functions
- ✅ COMPREHENSIVE: Create single comprehensive test files that cover entire feature areas
CREATE NEW TEST FILES ONLY WHEN
- ✅ Testing completely new subsystems with no existing related tests
- ✅ Isolated feature areas that don't fit logically into existing test structure
- ✅ Complex integration tests that require dedicated setup/teardown
AVOID
- ❌ Creating new test files for minor bug fixes - add to existing tests
- ❌ One test case per file - consolidate related functionality
- ❌ Duplicate test patterns across multiple files
- ❌ Scattered feature testing - keep related tests together
DEVELOPMENT WORKFLOW
- During Development/Bug Fixing: Temporary test files are acceptable for rapid iteration
- Near Completion: MANDATORY - Consolidate temporary tests into existing files
- Final Review: Remove temporary test files and ensure comprehensive coverage in main test files
CONSOLIDATION CHECKLIST
- Can this test be added to an existing
TEST_CASEin the same file? - Does an existing test file cover the same functional area?
- Would this test fit better as a sub-section of a comprehensive test?
- Are there duplicate test patterns that can be eliminated?
Why: Maintaining a clean, consolidated test suite:
- Easier maintenance - fewer files to manage and update
- Better organization - related functionality tested together
- Faster builds - fewer compilation units
- Cleaner repository - less file clutter
- Improved discoverability - easier to find existing test coverage
Blank Test Template
✅ CORRECT blank test file structure:
// Unit tests for general allocator functionality and integration tests
#include "test.h"
#include "FastLED.h"
using namespace fl;
TEST_CASE("New test - fill in") {
}
❌ WRONG patterns to avoid:
- Missing descriptive file header comment
- Missing proper includes (
test.h,FastLED.h) - Missing
using namespace fl;declaration - Empty or non-descriptive test case names
- Inconsistent formatting and spacing
Debugging and Stack Traces
Stack Trace Setup
The FastLED project supports enhanced debugging through stack trace functionality for crash analysis and debugging.
For Background Agents: Use the MCP server tool setup_stack_traces to automatically install and configure stack trace support:
# Via MCP server (recommended for background agents)
uv run mcp_server.py
# Then use setup_stack_traces tool with method: "auto"
Manual Installation:
Ubuntu/Debian:
sudo apt-get update
sudo apt-get install -y libunwind-dev build-essential cmake
CentOS/RHEL/Fedora:
sudo yum install -y libunwind-devel gcc-c++ cmake # CentOS/RHEL
sudo dnf install -y libunwind-devel gcc-c++ cmake # Fedora
macOS:
brew install libunwind
Available Stack Trace Methods
- LibUnwind (Recommended) - Enhanced stack traces with symbol resolution
- Execinfo (Fallback) - Basic stack traces using standard glibc
- Windows (On Windows) - Windows-specific debugging APIs
- No-op (Last resort) - Minimal crash handling
The build system automatically detects and configures the best available option.
Testing Stack Traces
cd tests
cmake . && make crash_test_standalone crash_test_execinfo
# Test libunwind version
./.build/bin/crash_test_standalone manual # Manual stack trace
./.build/bin/crash_test_standalone nullptr # Crash test
# Test execinfo version
./.build/bin/crash_test_execinfo manual # Manual stack trace
./.build/bin/crash_test_execinfo nullptr # Crash test
Using in Code
#include "tests/crash_handler.h"
int main() {
setup_crash_handler(); // Enable crash handling
// Your code here...
return 0;
}
Testing Infrastructure
Test Configuration: tests/cmake/TestConfiguration.cmake
- Defines test targets and dependencies
- Configures test execution parameters
- Sets up coverage and profiling
Test Execution:
bash test- Comprehensive test runner (preferred)uv run test.py- Python test interface- Individual test executables in
.build/bin/
Python Build System Commands for Testing
Unit Test Compilation:
# Fast Python API (default)
bash test --unit --verbose # Uses PCH optimization
# Disable PCH if needed
bash test --unit --no-pch --verbose
# Legacy CMake system (8x slower)
bash test --unit --legacy --verbose
Available Build Options:
--no-pch- Disable precompiled headers--clang- Use Clang compiler (recommended for speed)--clean- Force full rebuild--verbose- Show detailed compilation output--legacy- Use old CMake system (discouraged)
Memory Refresh Rule
🚨 ALL AGENTS: Read tests/AGENTS.md before concluding testing work to refresh memory about current test execution requirements and validation rules.