Commit 83d2ada
authored
Add support for instance group of type 'MODEL' (#107)
* Add support for instance group of type 'MODEL'
* Format
* Handle multi GPU cases when recording timestamps
* Address comment
* Use callback function to record timestamp for 'MODEL' kind
* Add missing #ifdef
* Update comment and if condition for input tensor memory alloc_perference
* Fix for cuda stream. Use separate cuda callback to capture timestamp
* Add comment to mention the possible timestamp issue
* For 'KIND_MODEL', use cuda events for compute_input_duration and use callback for compute_infer_duration
* Move the cudaLaunchHostFunc from RecordBackendTimestamp function
* Fix up naming
* Fix up
* Fix up atomic initialization
* Capture the timestamp after synchronization1 parent 550cf62 commit 83d2ada
1 file changed
+242
-65
lines changed
0 commit comments