Skip to content

AI: Introduce Perf Police and reusable code review and profiling skills - #776

Merged
Gaurav Sharma (bewithgaurav) merged 9 commits into
mainfrom
bewithgaurav/perf-police-skills
Sep 24, 2026
Merged

Gaurav Sharma (bewithgaurav) merged 9 commits into
mainfrom
bewithgaurav/perf-police-skills

Conversation

@bewithgaurav

@bewithgaurav Gaurav Sharma (bewithgaurav) commented Sep 10, 2026 •

Copy link
Copy Markdown
Collaborator

Work Item / Issue Reference

AB#44984
AB#48077

Summary

Add reusable Copilot review guidance while keeping general repository review separate from performance analysis.

  • Add a code-review skill sourced solely from the repository's Copilot instructions.
  • Add a performance-code-review skill covering native ownership, performance patterns, benchmark validity, and evidence-based findings.
  • Add an mssql-profiler skill for operating the existing profiler, including build checks, bounded scenarios, raw-data export, timelines, and paired comparisons.
  • Make Perf Police a thin custom agent that loads the performance skill instead of maintaining a duplicate checklist.
  • Accept the AI: PR-title prefix and align the PR template, contribution guidance, and release-note classification.

This PR does not change driver runtime behavior.

Validation

  • Validated agent and skill frontmatter, relative links, and the separation of general and performance guidance.
  • Exercised 18 cases against the actual PR-format workflow script, including existing prefixes, the new prefix, and description requirements.
  • Exercised the documented profiler CLI and Python examples and the 60-test profiler suite on macOS.
  • Cross-client skill discovery, reviewer behavior, and non-macOS profiler execution have not yet been evaluated.

Add a repository code-review skill containing the native performance review methodology and verified ownership, profiling, measurement, and runtime-evidence lessons. Expose it through a thin Perf Police custom agent so interactive reviews and other skill-capable reviewers share one rulebook. Correct obsolete execution-path assumptions and distinguish safety requirements from performance optimizations. Profiler operation remains documented in the existing profiler package; no profiler skill or runtime changes are included.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Derive the general code-review skill solely from repository Copilot instructions. Preserve the specialized methodology in performance-code-review and route the Perf Police agent there, keeping normal reviews independent of the performance checklist.

Add AI: to the accepted PR title prefixes and align the template, contributor instructions, PR creation guidance, and release-note classification. Reserve the category for AI tooling and development workflows rather than every AI-assisted fix. Existing title categories and description validation remain unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 10, 2026 15:08
@github-actions github-actions Bot added the pr-size: medium Moderate update size label Sep 10, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Changes are documentation/automation/agent-skill additions that are internally consistent, low risk, and align CI enforcement with updated contributor guidance.

Pull request overview

Adds reusable GitHub Copilot review guidance to the repo by introducing two Skills (general repository review and performance-focused review), and aligns repository contribution/PR-format documentation and automation to accept the AI: PR-title prefix.

Changes:

  • Add code-review and performance-code-review Skills under .github/skills/.
  • Introduce a thin “Perf Police” custom agent that delegates to the performance Skill.
  • Update PR title-prefix validation and contributor-facing guidance/templates to include AI: (and document PERF: consistently where applicable).
File summaries
File Description
CONTRIBUTING.md Documents AI: as an allowed PR title prefix and clarifies when to use it.
.github/workflows/pr-format-check.yml Updates CI enforcement to accept AI: as a valid PR title prefix.
.github/skills/performance-code-review/SKILL.md Adds an evidence-led performance review methodology Skill for native/perf-sensitive changes.
.github/skills/code-review/SKILL.md Adds a repository-wide review checklist Skill sourced from .github/copilot-instructions.md.
.github/PULL_REQUEST_TEMPLATE.MD Adds AI: to the PR template’s title prefix guidance.
.github/prompts/create-pr.prompt.md Updates the PR-creation prompt to include PERF: and AI: prefixes consistently.
.github/copilot-instructions.md Updates validation-gate guidance to include PERF: and AI: and defines intended AI: usage.
.github/agents/release-manager.agent.md Excludes AI: PRs from customer-facing release notes by default (unless clearly user-visible).
.github/agents/perf-police.agent.md Adds a Perf Police agent definition that points to the performance Skill as the single source of truth.
Review details
  • Files reviewed: 9/9 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Add an operator skill for the existing Python and native profiler, covering approved environments, build provenance, bounded scenarios, raw result export, timelines, controlled comparisons, and cleanup. Keep instrumented attribution separate from uninstrumented release latency and surface missing prerequisites instead of inventing results.

Link the profiler skill from performance review only when new measurements are requested. General repository review, the agent entry point, and the profiler implementation remain unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Sep 10, 2026 •

Copy link
Copy Markdown

📊 Code Coverage Report

🔥 Diff Coverage

100%


🎯 Overall Coverage

84%


📈 Total Lines Covered: 9054 out of 10697
📁 Project: mssql-python


Diff Coverage

Diff: main...HEAD, staged and unstaged changes

No lines with coverage information in this diff.


📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)
mssql_python.pybind.performance_counter.hpp: 0.7%
mssql_python.pybind.logger_bridge.cpp: 57.9%
mssql_python.pybind.ddbc_bindings.h: 64.1%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 78.4%
mssql_python.pybind.connection.connection_pool.cpp: 82.3%
mssql_python.pybind.connection.connection.cpp: 82.5%
mssql_python.logging.py: 86.2%
mssql_python.pooling.py: 90.1%
mssql_python.pybind.fetch_temporal.hpp: 92.1%

🔗 Quick Links

⚙️ Build Summary 📋 Coverage Details

View Azure DevOps Build

Browse Full Coverage Report

Copilot AI review requested due to automatic review settings September 10, 2026 17:49
@github-actions github-actions Bot added pr-size: large Substantial code update and removed pr-size: medium Moderate update size labels Sep 10, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are internally consistent across CI validation, contributor docs, templates, and the new skill/agent content, with no runtime-impacting modifications.

Review details
  • Files reviewed: 10/10 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@bewithgaurav Gaurav Sharma (bewithgaurav) changed the title AI: Add repository and performance review skills AI: Introduce Perf Police and reusable code review and profiling skills Sep 10, 2026
Copilot AI review requested due to automatic review settings September 16, 2026 05:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The profiler skill contains unresolved Release-build and architecture-command issues.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (2)

.github/skills/mssql-profiler/SKILL.md:202

  • The uninstrumented comparison command has the same single-config issue as the profiled command: build.sh does not configure CMAKE_BUILD_TYPE=Release, so --config Release does not make the resulting macOS/Linux binary a Release/-DNDEBUG build. This can make the shipped-latency control incomparable to the profiled build. Make the configure step explicitly select Release here as well.
(cd mssql_python/pybind && ENABLE_PROFILING=0 bash build.sh)

.github/skills/mssql-profiler/SKILL.md:209

  • The uninstrumented Windows comparison repeats the same hard-coded x64 target, so an ARM64 base/PR comparison cannot follow the documented procedure. Use the same explicit architecture variable as the profiling build rather than rebuilding a different target.
cmd /c "cd mssql_python\pybind && set ENABLE_PROFILING=0&& build.bat x64"

- **Files reviewed:** 10/10 changed files
- **Comments generated:** 2
- **Review effort level:** Lite
</details>

Comment thread .github/skills/mssql-profiler/SKILL.md
Comment thread .github/skills/mssql-profiler/SKILL.md
Copilot AI review requested due to automatic review settings September 23, 2026 06:08
@bewithgaurav
Gaurav Sharma (bewithgaurav) marked this pull request as ready for review September 23, 2026 06:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Profiler guidance has unresolved build-configuration, architecture-selection, and workload-description issues.

Review effort: Lite
Findings: 1 Medium severity · 1 Low severity

Open (2)

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

PR Performance Report

✅ No regression detected

No consistent slowdowns detected across all 2 environments.

0 IMPROVEMENTS 0 SLOWDOWNS 2/2 ENVIRONMENTS

Coverage: 2 of 2 environments completed. Advisory result; does not block merging.

Performance diagnostics

Phase times are inclusive diagnostics and must not be added together. They identify where measured time changed, not why it changed.

No affected phases or call-count changes were recorded.

All database tasks and timings

Unix / SQL Server 2022

Database task Before After Paired change Result
Connection opening 10.431 ms 10.955 ms +3.7% no signal
SELECT queries 1.079 ms 1.085 ms -1.8% no signal
Row insertion 32.269 ms 32.847 ms +1.3% no signal
Executemany inserts 155.144 ms 156.154 ms +0.5% no signal
Fetch-all queries 118.954 ms 119.983 ms +1.1% no signal
Row-by-row fetching 55.169 ms 54.843 ms -1.0% no signal
Batched row fetching 119.848 ms 121.969 ms +0.5% no signal
Transaction commit and rollback 109.114 ms 109.300 ms -0.2% no signal
Arrow row fetching 92.616 ms 94.678 ms +1.1% no signal
100,000-row insertion 439.031 ms 434.424 ms -0.5% no signal
Row fetching in batches of 100 172.873 ms 171.161 ms -1.0% no signal
Row fetching in batches of 10,000 121.550 ms 128.139 ms +1.5% no signal
Repeated positional queries 39.414 ms 39.861 ms +0.5% no signal
Repeated named-parameter queries 41.876 ms 41.840 ms -0.4% no signal
Legacy 100,000-row insertion 342.812 ms 345.611 ms -0.3% no signal
Insertion with explicit input sizes 469.972 ms 480.769 ms -0.1% no signal
Joined aggregation queries 176.220 ms 177.145 ms -0.2% no signal
Large joined-result fetching 182.676 ms 183.645 ms -1.2% no signal
1.2-million-row fetching 3383.113 ms 3409.753 ms +0.8% no signal
Common table expression queries 5.516 ms 5.408 ms -1.6% no signal

Unix / SQL Server 2025

Database task Before After Paired change Result
Connection opening 97.371 ms 97.571 ms +0.2% no signal
SELECT queries 1.091 ms 1.085 ms -0.1% no signal
Row insertion 34.348 ms 34.681 ms +0.5% no signal
Executemany inserts 152.377 ms 150.028 ms -1.1% no signal
Fetch-all queries 121.260 ms 121.644 ms -0.3% no signal
Row-by-row fetching 56.091 ms 55.926 ms -1.0% no signal
Batched row fetching 123.272 ms 123.043 ms +0.4% no signal
Transaction commit and rollback 114.430 ms 114.284 ms -0.3% no signal
Arrow row fetching 95.263 ms 93.805 ms -1.1% no signal
100,000-row insertion 461.075 ms 455.536 ms -1.4% no signal
Row fetching in batches of 100 174.695 ms 173.211 ms -1.0% no signal
Row fetching in batches of 10,000 143.334 ms 132.787 ms -3.9% no signal
Repeated positional queries 41.606 ms 41.534 ms +0.7% no signal
Repeated named-parameter queries 44.244 ms 43.800 ms -1.9% no signal
Legacy 100,000-row insertion 357.019 ms 354.901 ms -0.7% no signal
Insertion with explicit input sizes 484.636 ms 483.317 ms -0.3% no signal
Joined aggregation queries 159.372 ms 160.731 ms +0.7% no signal
Large joined-result fetching 188.984 ms 189.267 ms +0.1% no signal
1.2-million-row fetching 3454.780 ms 3454.957 ms -0.2% no signal
Common table expression queries 5.221 ms 5.192 ms -0.8% no signal
Build and measurement details

ADO build 177859

PR head: 993cf35222939e38701197413bb3b491e1eac597
Base: 1c14bb926048d900b38ccdcc14885122e177e5dc
Measured merge: 95cc6d4bbc44e2e3388b4e58da5dbe0eaab4160d

  • Unix / SQL Server 2022: Python 3.12.3, x86_64, SQL 16.0.4295.3; 5 paired comparisons and 1 warmup.
  • Unix / SQL Server 2025: Python 3.12.3, x86_64, SQL 17.0.5005.3; 5 paired comparisons and 1 warmup.

A consistent change requires more than 20% median paired movement, at least 1 ms between the median runtimes, and at least 80% of pairs exceeding the relative threshold in the same direction. A slowdown without enough pair agreement is reported as inconsistent.

The displayed change is the median of paired before-and-after ratios. It is not recalculated from the two displayed median runtimes.

Both revisions use profiling-enabled builds on the same agent and database, with alternating order and discarded warmups. Results are diagnostic and do not represent production-wheel latency.

Raw samples and logs are attached to the ADO run as profiler-* artifacts.

Copilot AI review requested due to automatic review settings September 24, 2026 06:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Resolve the four moderate issues involving profiler build/architecture handling and Perf Police PR input support.

Review effort: Lite
Findings: 1 Medium severity · 1 Low severity

Open (2)

gargsaumya
gargsaumya previously approved these changes Sep 24, 2026
Copilot AI review requested due to automatic review settings September 24, 2026 07:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved profiler build-configuration and architecture issues, plus a stale skill reference, remain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Low severity

Open (1)
Resolved since last review (2)

Comment thread .github/skills/code-review/SKILL.md Outdated
Clarify the source of repository-specific rules for the skill.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The profiler guidance has unresolved build-configuration and architecture issues that can invalidate profiling results.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Low severity

Open (1)
Resolved since last review (1)

Comment thread .github/skills/code-review/SKILL.md
Copilot AI review requested due to automatic review settings September 24, 2026 08:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Moderate issues remain in profiler build/architecture guidance and duplicated review-skill wording.

Review effort: Lite
Findings: None

Resolved since last review (1)

@bewithgaurav
Gaurav Sharma (bewithgaurav) merged commit 527c37f into main Sep 24, 2026
29 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-size: large Substantial code update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants