esptest.testcase package

class esptest.testcase.ResultDetail(type, context='', params=<factory>, result=<factory>, brief_message='', started_at=None, finished_at=None, file='')

Bases: object

ResultDetail is a class that records the result of a performance test.

Each accumulated record corresponds to one TestResultDetail database row and can be dumped as JSON / JSONL for batch import into the database.

Database schema (TestResultDetail) that records the target:

params:        dict[str, Any]           (JSONB, required)
result:        dict[str, Any]           (JSONB, required)
duration:      int | None
logs:          list[dict[str, Any]] | None
brief_message: str | None
started_at:    datetime | None
finished_at:   datetime | None

Parameters:

• type (str)

• context (str)

• params (Dict[str, Any])

• result (Dict[str, Any])

• brief_message (str)

• started_at (str | None)

• finished_at (str | None)

• file (str)

type: str

context: str = ''

params: Dict[str, Any]

result: Dict[str, Any]

brief_message: str = ''

started_at: str | None = None

finished_at: str | None = None

file: str = ''

to_dict()

Return type:

Dict[str, Any]

classmethod from_dict(data)

Parameters:

data (Dict[str, Any])

Return type:

ResultDetail

classmethod load_json(path)

Parameters:

path (str | Path)

Return type:

ResultDetail

to_json(indent=2)

Parameters:

indent (int | None)

Return type:

str

to_text()

Return type:

str

to_markdown()

Return type:

str

save_json(path, indent=2)

Parameters:

• path (str | Path)

• indent (int | None)

Return type:

Path

save_text(path)

Parameters:

path (str | Path)

Return type:

Path

save_markdown(path)

Parameters:

path (str | Path)

Return type:

Path

class esptest.testcase.TestCaseResult(name: str, classname: str = '', status: str = 'passed', duration: Optional[float] = None, message: Optional[str] = None, failure_type: Optional[str] = None, stdout: Optional[str] = None, stderr: Optional[str] = None, properties: Dict[str, str] = <factory>, logs: Optional[List[Dict[str, Any]]] = None, result_detail_files: List[str] = <factory>, result_details: List[esptest.testcase.result.ResultDetail] = <factory>, started_at: Optional[str] = None)

Bases: object

Parameters:

• name (str)

• classname (str)

• status (str)

• duration (float | None)

• message (str | None)

• failure_type (str | None)

• stdout (str | None)

• stderr (str | None)

• properties (Dict[str, str])

• logs (List[Dict[str, Any]] | None)

• result_detail_files (List[str])

• result_details (List[ResultDetail])

• started_at (str | None)

name: str

classname: str = ''

status: str = 'passed'

duration: float | None = None

message: str | None = None

failure_type: str | None = None

stdout: str | None = None

stderr: str | None = None

properties: Dict[str, str]

logs: List[Dict[str, Any]] | None = None

result_detail_files: List[str]

result_details: List[ResultDetail]

started_at: str | None = None

add_result_detail(detail, file_name='')

Attach a ResultDetail object directly to this case.

The object is kept in result_details (in memory). When file_name (a path relative to the report/log directory) is given, it is stored on the detail (detail.file) and appended to result_detail_files so the report can reference the saved file. Writing the file itself is left to the caller (e.g. detail.save_json). Returns the same object for chaining.

Parameters:

• detail (ResultDetail)

• file_name (str)

Return type:

ResultDetail

class esptest.testcase.TestCaseStatus

Bases: object

PASSED = 'passed'

FAILED = 'failed'

ERROR = 'error'

SKIPPED = 'skipped'

class esptest.testcase.TestSuiteResult(name: str, test_cases: List[esptest.testcase.result.TestCaseResult] = <factory>, properties: Dict[str, str] = <factory>, timestamp: Optional[str] = None, package: Optional[str] = None, hostname: Optional[str] = None, file: Optional[str] = None)

Bases: object

Parameters:

• name (str)

• test_cases (List[TestCaseResult])

• properties (Dict[str, str])

• timestamp (str | None)

• package (str | None)

• hostname (str | None)

• file (str | None)

name: str

test_cases: List[TestCaseResult]

properties: Dict[str, str]

timestamp: str | None = None

package: str | None = None

hostname: str | None = None

file: str | None = None

property tests: int

property failures: int

property errors: int

property skipped: int

property time: float

class esptest.testcase.TestSuitesResult(name: str = 'testsuites', test_suites: List[esptest.testcase.result.TestSuiteResult] = <factory>, properties: Dict[str, str] = <factory>)

Bases: object

Parameters:

• name (str)

• test_suites (List[TestSuiteResult])

• properties (Dict[str, str])

name: str = 'testsuites'

test_suites: List[TestSuiteResult]

properties: Dict[str, str]

property tests: int

property failures: int

property errors: int

property skipped: int

property time: float

class esptest.testcase.EspTestCase(methodName='runTest')

Bases: TestCase

A unittest.TestCase that streams results into an xUnit report.

Subclasses only need to set xunit_log_dir (and optionally target / config / xunit_suite_name) and write plain test_* methods. Each case is opened in setUp() and closed in tearDown() with the result derived from the runner outcome.

Under pytest, the bind_case_context fixture from esptest.pytest_plugin can inject target / config / xunit_log_dir onto the class automatically.

To fold several classes (or a whole pytest session) into a single report, assign a shared XunitLogger to xunit_logger before setUpClass() runs; the class then reuses that logger and does not close it (the owner is responsible for that).

target: str = 'unknown'

chip target, used in the default case_id()

config: str = 'Default'

build/config label, used in the default case_id()

xunit_log_dir: str = ''

directory the xUnit report (and per-case artifacts) are written under

xunit_suite_name: str = ''

xUnit suite name; defaults to the class name when empty

xunit_logger: XunitLogger | None = None

the logger; created in setUpClass(), or set externally to share one

case_id()

xUnit/JUnit case id, <target>.<config>.<method_name> by default.

Return type:

str

classmethod setUpClass()

Hook method for setting up class fixture before running tests in the class.

Return type:

None

classmethod tearDownClass()

Hook method for deconstructing the class fixture after running all tests in the class.

Return type:

None

setUp()

Hook method for setting up the test fixture before exercising it.

Return type:

None

tearDown()

Hook method for deconstructing the test fixture after testing it.

Return type:

None

class esptest.testcase.XunitLogger(path, suite_name='test-suite', file_name='XUNIT_RESULT.xml', flush_interval=2.0, timestamp=None, package='esptest', hostname='build-33769991-project-1187477-esp-test-utils', std_head_len=4096, std_tail_len=4096)

Bases: object

Parameters:

• path (str | Path)

• suite_name (str)

• file_name (str)

• flush_interval (float)

• timestamp (str | None)

• package (str)

• hostname (str)

• std_head_len (int)

• std_tail_len (int)

property test_suite: TestSuiteResult

property has_running_case: bool

property current_test_case: TestCaseResult | None

set_config(config)

Parameters:

config (Dict[str, str])

Return type:

None

begin_case(case_id, classname='', category=None)

Parameters:

• case_id (str)

• classname (str)

• category (str | None)

Return type:

None

add_sys_out(message)

Parameters:

message (str)

Return type:

None

add_sys_err(message)

Parameters:

message (str)

Return type:

None

add_failure(message='Fail Reason Not Set', fail_type='unknown')

Parameters:

• message (str)

• fail_type (str)

Return type:

None

add_error(message='Fail Reason Not Set')

Parameters:

message (str)

Return type:

None

add_skipped(message='')

Parameters:

message (str)

Return type:

None

clear_failures()

Return type:

None

end_case(result=True, message='', failure_type='')

Parameters:

• result (bool)

• message (str)

• failure_type (str)

Return type:

Path

flush(force=False)

Parameters:

force (bool)

Return type:

Path

close(message='Test case interrupted before end_case')

Parameters:

message (str)

Return type:

Path

get_cur_case_result()

Return type:

Tuple[bool, str]

get_cur_case_id()

Return type:

str

esptest.testcase.generate_xunit_xml(test_suites)

Parameters:

test_suites (TestSuitesResult)

Return type:

str

esptest.testcase.get_case_result_from_outcome(test_case)

Read pass/fail and a brief message for the current test method.

Meant to be called from tearDown and works with both runners, whose outcome objects differ:

• unittest’s _Outcome keeps (test, exc_info) tuples on .errors, populated before tearDown runs (result.failures is only filled in afterwards, so it cannot be relied on here).

• pytest keeps an ExceptionInfo list on _outcome.result._excinfo.

Returns (passed, message); message is empty when the case passed.

Parameters:

test_case (TestCase)

Return type:

Tuple[bool, str]

esptest.testcase.parse_xunit_xml(xml_or_path, base_dir=None, load_result_details=True)

Parameters:

• xml_or_path (str | Path)

• base_dir (str | Path | None)

• load_result_details (bool)

Return type:

TestSuitesResult

esptest.testcase.save_xunit_xml(test_suites, path)

Parameters:

• test_suites (TestSuitesResult)

• path (str | Path)

Return type:

Path

Submodules

• esptest.testcase.result module
  • ResultDetail
    • ResultDetail.type
    • ResultDetail.context
    • ResultDetail.params
    • ResultDetail.result
    • ResultDetail.brief_message
    • ResultDetail.started_at
    • ResultDetail.finished_at
    • ResultDetail.file
    • ResultDetail.to_dict()
    • ResultDetail.from_dict()
    • ResultDetail.load_json()
    • ResultDetail.to_json()
    • ResultDetail.to_text()
    • ResultDetail.to_markdown()
    • ResultDetail.save_json()
    • ResultDetail.save_text()
    • ResultDetail.save_markdown()
  • TestCaseStatus
    • TestCaseStatus.PASSED
    • TestCaseStatus.FAILED
    • TestCaseStatus.ERROR
    • TestCaseStatus.SKIPPED
  • TestCaseResult
    • TestCaseResult.name
    • TestCaseResult.classname
    • TestCaseResult.status
    • TestCaseResult.duration
    • TestCaseResult.message
    • TestCaseResult.failure_type
    • TestCaseResult.stdout
    • TestCaseResult.stderr
    • TestCaseResult.properties
    • TestCaseResult.logs
    • TestCaseResult.result_detail_files
    • TestCaseResult.result_details
    • TestCaseResult.started_at
    • TestCaseResult.add_result_detail()
  • TestSuiteResult
    • TestSuiteResult.name
    • TestSuiteResult.test_cases
    • TestSuiteResult.properties
    • TestSuiteResult.timestamp
    • TestSuiteResult.package
    • TestSuiteResult.hostname
    • TestSuiteResult.file
    • TestSuiteResult.tests
    • TestSuiteResult.failures
    • TestSuiteResult.errors
    • TestSuiteResult.skipped
    • TestSuiteResult.time
  • TestSuitesResult
    • TestSuitesResult.name
    • TestSuitesResult.test_suites
    • TestSuitesResult.properties
    • TestSuitesResult.tests
    • TestSuitesResult.failures
    • TestSuitesResult.errors
    • TestSuitesResult.skipped
    • TestSuitesResult.time
• esptest.testcase.unittest_case module
  • get_case_result_from_outcome()
  • EspTestCase
    • EspTestCase.target
    • EspTestCase.config
    • EspTestCase.xunit_log_dir
    • EspTestCase.xunit_suite_name
    • EspTestCase.xunit_logger
    • EspTestCase.case_id()
    • EspTestCase.setUpClass()
    • EspTestCase.tearDownClass()
    • EspTestCase.setUp()
    • EspTestCase.tearDown()
• esptest.testcase.xunit module
  • generate_xunit_xml()
  • save_xunit_xml()
  • parse_xunit_xml()
  • XunitLogger
    • XunitLogger.test_suite
    • XunitLogger.has_running_case
    • XunitLogger.current_test_case
    • XunitLogger.set_config()
    • XunitLogger.begin_case()
    • XunitLogger.add_sys_out()
    • XunitLogger.add_sys_err()
    • XunitLogger.add_failure()
    • XunitLogger.add_error()
    • XunitLogger.add_skipped()
    • XunitLogger.clear_failures()
    • XunitLogger.end_case()
    • XunitLogger.flush()
    • XunitLogger.close()
    • XunitLogger.get_cur_case_result()
    • XunitLogger.get_cur_case_id()