[Step 1/11] Parsing and validating input arguments [ INFO ] Parsing input parameters usage: benchmark_app [-h [HELP]] [-i PATHS_TO_INPUT [PATHS_TO_INPUT ...]] -m PATH_TO_MODEL [-d TARGET_DEVICE] [-hint {throughput,tput,cumulative_throughput,ctput,latency,none}] [-niter NUMBER_ITERATIONS] [-max_irate MAXIMUM_INFERENCE_RATE] [-t TIME] [-b BATCH_SIZE] [-shape SHAPE] [-data_shape DATA_SHAPE] [-layout LAYOUT] [-extensions EXTENSIONS] [-c PATH_TO_CLDNN_CONFIG] [-cdir CACHE_DIR] [-lfile [LOAD_FROM_FILE]] [-api {sync,async}] [-nireq NUMBER_INFER_REQUESTS] [-nstreams NUMBER_STREAMS] [-inference_only [INFERENCE_ONLY]] [-no_warmup] [-infer_precision INFER_PRECISION] [-ip {bool,f16,f32,f64,i8,i16,i32,i64,u8,u16,u32,u64}] [-op {bool,f16,f32,f64,i8,i16,i32,i64,u8,u16,u32,u64}] [-iop INPUT_OUTPUT_PRECISION] [--mean_values [R,G,B]] [--scale_values [R,G,B]] [-nthreads NUMBER_THREADS] [-pin {YES,NO}] [-latency_percentile LATENCY_PERCENTILE] [-report_type {no_counters,average_counters,detailed_counters}] [-report_folder REPORT_FOLDER] [-json_stats [JSON_STATS]] [-pc [PERF_COUNTS]] [-pcsort {no_sort,sort,simple_sort}] [-pcseq [PCSEQ]] [-exec_graph_path EXEC_GRAPH_PATH] [-dump_config DUMP_CONFIG] [-load_config LOAD_CONFIG] Options: -h [HELP], --help [HELP] Show this help message and exit. -i PATHS_TO_INPUT [PATHS_TO_INPUT ...], --paths_to_input PATHS_TO_INPUT [PATHS_TO_INPUT ...] Optional. Path to a folder with images and/or binaries or to specific image or binary file.It is also allowed to map files to model inputs: input_1:file_1/dir1,file_2/dir2,input_4:file_4/dir4 input_2:file_3/dir3 Currently supported data types: bin, npy. If OPENCV is enabled, this functionalityis extended with the following data types: bmp, dib, jpeg, jpg, jpe, jp2, png, pbm, pgm, ppm, sr, ras, tiff, tif. -m PATH_TO_MODEL, --path_to_model PATH_TO_MODEL Required. Path to an .xml/.onnx file with a trained model or to a .blob file with a trained compiled model. -d TARGET_DEVICE, --target_device TARGET_DEVICE Optional. Specify a target device to infer on (the list of available devices is shown below). Default value is CPU. Use '-d HETERO:' format to specify HETERO plugin. Use '-d MULTI:' format to specify MULTI plugin. The application looks for a suitable plugin for the specified device. -hint {throughput,tput,cumulative_throughput,ctput,latency,none}, --perf_hint {throughput,tput,cumulative_throughput,ctput,latency,none} Optional. Performance hint (latency or throughput or cumulative_throughput or none). Performance hint allows the OpenVINO device to select the right model-specific settings. 'throughput': device performance mode will be set to THROUGHPUT. 'cumulative_throughput': device performance mode will be set to CUMULATIVE_THROUGHPUT. 'latency': device performance mode will be set to LATENCY. 'none': no device performance mode will be set. Using explicit 'nstreams' or other device-specific options, please set hint to 'none' -niter NUMBER_ITERATIONS, --number_iterations NUMBER_ITERATIONS Optional. Number of iterations. If not specified, the number of iterations is calculated depending on a device. Set -niter 0 to compile the model and exit without running inference (useful for models with dynamic shapes that would otherwise require -shape, -data_shape, or -i). -max_irate MAXIMUM_INFERENCE_RATE, --maximum_inference_rate MAXIMUM_INFERENCE_RATE Optional. Maximum inference rate by frame per second. If not specified, default value is 0, the inference will run at maximium rate depending on a device capabilities. Tweaking this value allow better accuracy in power usage measurement by limiting the execution. -t TIME, --time TIME Optional. Time in seconds to execute topology. -api {sync,async}, --api_type {sync,async} Optional. Enable using sync/async API. When hint is throughput, default value is async. When hint is latency, default value is sync. -json_stats [JSON_STATS], --json_stats [JSON_STATS] Optional. Enables JSON-based statistics output (by default reporting system will use CSV format). Should be used together with -report_folder option. Input shapes: -b BATCH_SIZE, --batch_size BATCH_SIZE Optional. Batch size value. If not specified, the batch size value is determined from Intermediate Representation -shape SHAPE Optional. Set shape for input. For example, "input1[1,3,224,224],input2[1,4]" or "[1,3,224,224]" in case of one input size. This parameter affect model Parameter shape, can be dynamic. For dynamic dimesions use symbol `?`, `-1` or range `low.. up`. -data_shape DATA_SHAPE Optional. Optional if model shapes are all static (original ones or set by -shape).Required if at least one input shape is dynamic and input images are not provided.Set shape for input tensors. For example, "input1[1,3,224,224][1,3,448,448],input2[1,4][1,8]" or "[1,3,224,224][1,3,448,448] in case of one input size. -layout LAYOUT Optional. Prompts how model layouts should be treated by application. For example, "input1[NCHW],input2[NC]" or "[NCHW]" in case of one input size. Advanced options: -extensions EXTENSIONS, --extensions EXTENSIONS Optional. Path or a comma-separated list of paths to libraries (.so or .dll) with extensions. -c PATH_TO_CLDNN_CONFIG, --path_to_cldnn_config PATH_TO_CLDNN_CONFIG Optional. Required for GPU custom kernels. Absolute path to an .xml file with the kernels description. -cdir CACHE_DIR, --cache_dir CACHE_DIR Optional. Enable model caching to specified directory -lfile [LOAD_FROM_FILE], --load_from_file [LOAD_FROM_FILE] Optional. Loads model from file directly without read_model. -nireq NUMBER_INFER_REQUESTS, --number_infer_requests NUMBER_INFER_REQUESTS Optional. Number of infer requests. Default value is determined automatically for device. -nstreams NUMBER_STREAMS, --number_streams NUMBER_STREAMS Optional. Number of streams to use for inference on the CPU/GPU (for HETERO and MULTI device cases use format :,: or just ). Default value is determined automatically for a device. Please note that although the automatic selection usually provides a reasonable performance, it still may be non - optimal for some cases, especially for very small models. Also, using nstreams>1 is inherently throughput-oriented option, while for the best-latency estimations the number of streams should be set to 1. See samples README for more details. -inference_only [INFERENCE_ONLY], --inference_only [INFERENCE_ONLY] Optional. If true inputs filling only once before measurements (default for static models), else inputs filling is included into loop measurement (default for dynamic models) -no_warmup Optional. Skip warmup inference. Useful for benchmarking purposes in simulated environments. Otherwise, not recommended. -infer_precision INFER_PRECISION Optional. Specifies the inference precision. Example #1: '-infer_precision bf16'. Example #2: '-infer_precision CPU:bf16,GPU:f32' -exec_graph_path EXEC_GRAPH_PATH, --exec_graph_path EXEC_GRAPH_PATH Optional. Path to a file where to store executable graph information serialized. Preprocessing options: -ip {bool,f16,f32,f64,i8,i16,i32,i64,u8,u16,u32,u64}, --input_precision {bool,f16,f32,f64,i8,i16,i32,i64,u8,u16,u32,u64} Optional. Specifies precision for all input layers of the model. -op {bool,f16,f32,f64,i8,i16,i32,i64,u8,u16,u32,u64}, --output_precision {bool,f16,f32,f64,i8,i16,i32,i64,u8,u16,u32,u64} Optional. Specifies precision for all output layers of the model. -iop INPUT_OUTPUT_PRECISION, --input_output_precision INPUT_OUTPUT_PRECISION Optional. Specifies precision for input and output layers by name. Example: -iop "input:f16, output:f16". Notice that quotes are required. Overwrites precision from ip and op options for specified layers. --mean_values [R,G,B] Optional. Mean values to be used for the input image per channel. Values to be provided in the [R,G,B] format. Can be defined for desired input of the model, for example: "--mean_values data[255,255,255],info[255,255,255]". The exact meaning and order of channels depend on how the original model was trained. Applying the values affects performance and may cause type conversion --scale_values [R,G,B] Optional. Scale values to be used for the input image per channel. Values are provided in the [R,G,B] format. Can be defined for desired input of the model, for example: "--scale_values data[255,255,255],info[255,255,255]". The exact meaning and order of channels depend on how the original model was trained. If both --mean_values and --scale_values are specified, the mean is subtracted first and then scale is applied regardless of the order of options in command line. Applying the values affects performance and may cause type conversion Device-specific performance options: -nthreads NUMBER_THREADS, --number_threads NUMBER_THREADS Number of threads to use for inference on the CPU (including HETERO and MULTI cases). -pin {YES,NO}, --infer_threads_pinning {YES,NO} Optional. Enable threads->cores pinning for CPU-involved inference. Statistics dumping options: -latency_percentile LATENCY_PERCENTILE, --latency_percentile LATENCY_PERCENTILE Optional. Defines the percentile to be reported in latency metric. The valid range is [1, 100]. The default value is 50 (median). -report_type {no_counters,average_counters,detailed_counters}, --report_type {no_counters,average_counters,detailed_counters} Optional. Enable collecting statistics report. "no_counters" report contains configuration options specified, resulting FPS and latency. "average_counters" report extends "no_counters" report and additionally includes average PM counters values for each layer from the model. "detailed_counters" report extends "average_counters" report and additionally includes per-layer PM counters and latency for each executed infer request. -report_folder REPORT_FOLDER, --report_folder REPORT_FOLDER Optional. Path to a folder where statistics report is stored. -pc [PERF_COUNTS], --perf_counts [PERF_COUNTS] Optional. Report performance counters. -pcsort {no_sort,sort,simple_sort}, --perf_counts_sort {no_sort,sort,simple_sort} Optional. Report performance counters and analysis the sort hotpoint opts. sort: Analysis opts time cost, print by hotpoint order no_sort: Analysis opts time cost, print by normal order simple_sort: Analysis opts time cost, only print EXECUTED opts by normal order -pcseq [PCSEQ], --pcseq [PCSEQ] Optional. Report latencies for each shape in -data_shape sequence. -dump_config DUMP_CONFIG Optional. Path to JSON file to dump OpenVINO parameters, which were set by application. -load_config LOAD_CONFIG Optional. Path to JSON file to load custom OpenVINO parameters. Please note, command line parameters have higher priority then parameters from configuration file. Example 1: a simple JSON file for HW device with primary properties. { "CPU": {"NUM_STREAMS": "3", "PERF_COUNT": "NO"} } Example 2: a simple JSON file for meta device(AUTO/MULTI) with HW device properties. { "AUTO": { "PERFORMANCE_HINT": "THROUGHPUT", "PERF_COUNT": "NO", "DEVICE_PROPERTIES": "{CPU:{INFERENCE_PRECISION_HINT:f32,NUM_STREAMS:3},GPU:{INFERENCE_PRECISION_HINT:f32,NUM_STREAMS:5}}" } } hwloc/linux: failed to find sysfs cpu topology directory, aborting linux discovery. Available target devices: CPU