lp:~kparichay/nnstreamer/+git/trunk

Author: Parichay Kapoor
Author Date: 2022-02-03 03:12:24 UTC

[filter] Support complex output types for pytorch

This patch adds support for complex output types for pytorch, where the
complex output is serialized before being passed out to the gstreamer
based element - tensorfilter.

V2:
- Added more comprehensive error messages
- Added indentation fix

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2022-01-27 08:44:04 UTC

[filter] Bug fix for pytorch filter with gpu

ptyorch filter when working with gpu backend had bug when converting
between the devices. The tensors returned by the conversion were not
catched. This patch provides the corresponding fix.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Yongjoo Ahn
Author Date: 2021-12-27 13:31:16 UTC

[test/pytorch] Add ssat test for multiple input/output tensor model

- Add a simple test for pytorch model whose input and output are multiple tensors

Signed-off-by: Yongjoo Ahn <yongjoo1.ahn@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-11-03 05:40:17 UTC

[tensorfilter] Set default framework when no priority

tensor_filter auto detection of framework takes in the extension
and for tflite extension tries to identify the framework using
a priority set with the configuration.
however, in case, the priority string is NULL, no tensor_filter
framework is set although multiple tensor_filters supporting
this extension can exist.

This patch sets a default framework for tflite extension which is
overwritten based on the priority if priority exists.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-10-27 02:49:26 UTC

[dist/debian] Package c-api headers

Deiban packaging of nnstreamer packages nnstreamer and capi-nnstreamer shared libraries
The corresponding headers are packaged as part of dev package. However, the headers
of c-api are not packaged. This patch packages the missing c-api headers.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-09-26 04:10:26 UTC

[docs] Update getting-started with minimal requirement

Update getting-started.md for ubuntu to include minimal requirement
required to build nnstreamer with default configuration

V2:
Added ninja-build as dependency as nnstreamer ppa meson does not
list it as a dependency

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Wook Song
Author Date: 2020-09-15 08:28:44 UTC

[Meson] Add missing dependency in flatbuffer-support declaration

This patch adds the missing dependency in the flatbuffer-support feature
declaration.

Signed-off-by: Wook Song <wook16.song@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-09-08 10:36:20 UTC

[hw/accl] Move hardware dependent code out of tensor_filter

As per the comments received in #2651, this patch moves the hardware
dependent code out of the tensor filter.

V2:
Also updated bugfix for linux (non android, non tizen) arm build
for proper header inclusion.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-09-08 07:51:48 UTC

[single/test] Add unittest for allocate_in_invoke free with single API

This patch adds unittest for single API using tensor filters custom framework
which support allocate_in_invoke
Further, adds negative and positive unittests verifying that the output
memory can be allocated by the user or when the single API handle is closed

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-08-14 12:51:01 UTC

[filter] Independency on order of the properties

This patch adds independence on the order of setting the properties
Properties of tensor filter which depends on the framework can now be set before setting the framework as well
When setting the framework, these properties are set again from the cache

Resolves #2650

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-08-18 08:08:05 UTC

[ext] Update checkAvailability to use runtime checks

Add a default implementation for frameworks to use for checkAvailability which includes
checking availability of certain backends (like NEON) at runtime.
For framework version 1, the same has been updated at gst_tensor_filter_check_hw_availability()

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-08-14 12:05:25 UTC

[single] Bug fix for setting accelerator

Bug fix for setting accelerator in the C-API.
When accelerator is set after the framework, tensor_filter does not know the framework
to check its version and store that information appropriately.
This patch changes the order of setting this information.
However, this bug still exists for pipeline API.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-08-13 02:42:59 UTC

[meson] Disable pytorch+caffe2 support for ubuntu <18.04

Disable support for pytorch and caffe2 filters for ubuntu version < 18.04
This is because pytorch and caffe2 are not independent of the protobuf version
they are compiled with.
With 18.04 ubuntu, 1.6.0 version of pytorch/caffe2 is compiled with protobuf version 3.12.3
However, with 16.04, 1.1.0 version of pytorch/caffe2 is compiled with protobuf version 3.6.1
(compiling 1.1.0 version of pytorch/caffe2 with protobuf version >3.6 fails)

So, for using pytorch/caffe2 on ubuntu 16.04, use old version (1.1.0) of pytorch/caffe2, and
build nnstreamer locally with protobuf version 3.6.1

See also #2411

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-07-29 04:29:10 UTC

[nnfw] Optimize nnfw invoke

This patch caches the type for the input and output data tensor
rather than calculating it at every invoke call.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-07-27 04:26:00 UTC

[tflite] Minimize invoke critical path

Minimize invoke critical path for tflite extension:
1. Cache input and output tensor ptrs out of the invoke to the location where extension
stores input and output tensors info
2. Verification of output size also moved out of invoke as the output tensor memory is
allocated internally by nnstreamer and its size is verified by each of the single API
and by tensor_filter.c for pipeline

V2:
Added bug fix to basic unittests for tflite reload exposed with the changes proposed in this PR

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-08-04 08:43:11 UTC

[Dist/Tizen] Package unittests

Add packaging of unittests for Tizen distribution

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-07-23 12:33:46 UTC

[Single] Add unittest to measure invoke latency by single API

Added unittest to measure the invoke latency by single API.
This latency is measured for tensorflow-lite and nnfw-runtime filters.
The latency is measured by running the invoke with single API and running
the invoke over the same data with the same model using filter API directly.
The difference in average execution time is reported as the latency introduced
by the single API invoke per call.
The invoke is called 1000 times after a warmup where the output is also verified.

V2:
Direct latency is the invoke duration of the library call
Latency induced by the SINGLE API is inclusive of the overhead by the API and the extension
Overhead induced by the extension is the overhead by the invoke function in the extension

V3:
Sleep for 30sec between consecutive latency tests for different frameworks for cooldown

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-08-03 10:58:11 UTC

[c-api] Add unlikely for improved branch prediction

Add G_UNLIKELY at all the error checks for invoke to improve branch predicition

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-07-28 04:16:36 UTC

[filter/single] Optimize invoke path for tensor filter single

Tensor filter single performs various check on invoke
However, these checks are done on properties which remain static after open and first invoke
So, these checks are now performed on the first invoke only
Further, some checks were being performed multiple times, which have now been removed.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-07-28 07:23:08 UTC

[tensor] Update GstTensorMemory coherent with ml_tensors_data_s

One of the efforts to minimize API latency is to avoid creating data wrappers in
API and internally in nnstreamer and avoid its copying. Rather just interpret
the API data structure as internal definition.
This requires internal and external structures to be coherent.
This commit makes GstTensorMemory coherent with ml_tensor_data_s by removing
`tensor_type type` from GstTensorMemory.

type can be remove from GstTensorMemory as GstTensorMemory is used internally
and its usage is always supported with GstTensorInfo which contains the type information.
Correspondingly update the existing elements and extensions which use GstTensorMemory
to use the GstTensorInfo type information.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-07-28 03:45:28 UTC

[single] Optimize invoke critical path

Remove setup gstTensorMemory in each invoke call
rather, pre-set the tensor memory when input dimensions are set
and re-use that setup tensor memory wrapper each time

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-07-29 08:26:49 UTC

[filter] Buf fix of setting num_tensors to 0

In case, empty string is provided for the property,
tensor filter set_property() sets the number of tensors to 0 in
its private info.

Now with this patch, updating the num_tensors to non-zero is allowed.
However, setting to 0 is not.

Resolves #2594

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-07-29 07:52:38 UTC

[C-API] Allocate memory at once for tensor in tensors

This patch changes the semantics of memory allocation for tensor data.
Tensor data is now allocated for all the tensor in tensors.
This reduces the multiple calls to malloc/free to 1, thus reducing the latency.
The final location of the memory in the tensor data is set with the offset.
This updates the memory allocated ml_tensors_data_create() and memory allocated
internally in single API tensor filter.
Correspondongly ml_tensors_data_destroy is also correspondingly update.

Note that this does not update the memory allocation by the pipeline API elements.
The tensor filter in the pipeline API retains its original behavior.

This patch exposes 2 bugs :

1. Memory allocated by the tensor filters inside them (which support allocate_in_invoke)
is not freed appropriately. When using the single API, this memory is wrapped with ml_tensors_data
and freed wtih ml_tensors_data_destroy. However, these memories allocated by the tensor filter itself
must be destroyed with destroyer of the tensor filter extension (issue #2593).
This will be fixed in a different patch.

2. Single API sets the input/output tensors info provided by the user.
However, it sets all the info with empty string which is might not be even provided.
tensor_filter set_property() ignores empty properties for most of the times.
However it updates the number of tensors even with empty properties (issue #2594).
The failing unittest (nnstreamer_capi_singleshot.set_input_info_success_02) has been
fixed for now. A proper fix to the internals will be done in a different patch.

**Self evaluation:**
1. Build test: [x]Passed [ ]Failed [ ]Skipped
2. Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-07-28 03:56:24 UTC

[single] Memory leak bug fix

Add memory leak bug fix
When output ml_tensors_data_h fails to allocate, the already computed output must be freed before return

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-07-27 02:00:15 UTC

[unittest] Bug fix for c-api unittest

Bug fix for c-api unittest when tensorflow-lite is not present
Bug of destroy info without create in invoke_01

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-07-24 01:45:45 UTC

[single] Enable setting timeout to 0

Enable setting timeout to 0
timeout 0 now has significance and re-setting timeout to 0 should be allowed

Related issue: #2568

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-07-16 09:59:05 UTC

[caffe2] Warning fix based for updated caffe2 v1.6.0-rc1

Update caffe2 extension to remove warning based on api update
for version 1.6.0-rc1

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-05-18 18:13:51 UTC

[single] Add missing gstTensorFilterFramework v1 support

Add missing support for gstTensorFilterFramework in checking framework availability
for a given backend accelerator/hardware

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-05-15 10:28:47 UTC

[subplugins] Update subplugins accl support

Update subplugin's accelerators support as per updated rules for auto/default

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-05-12 21:18:20 UTC

[filter] Handle auto/default semantics for tensor_filter v1

Patch to handle auto/default semantics for tensor_filter v1
getFrameworkInfo adds two more parameters - accl_default and accl_auto with the same characteristics as v0
Passed user input is parsed and if auto/default is used, corresponding accelerators are set and used.

This resolves #2320 for tensor_filter v1

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-05-14 06:31:11 UTC

[hw/accl] Add CPU_SIMD as a backend option

Added CPU_SIMD as a backend option
CPU_NEON will be alias for CPU_SIMD in enum but different string representation

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-05-13 03:33:31 UTC

[single] Add implementation to support for hw accelerator

single API supported hw accelerator for the NNFW
this hw was used to check if it is supported by the framework
however this hw was never passed to the framework to use
this patch sovles this issue

resolves #2374

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-05-12 21:05:02 UTC

[tflite] bug fix for tflite unittest

unittest of tflite compared output with grep "exp"
however some expressions could be prefix of different exp
this could lead to false positives
added line end marker

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-05-11 19:40:08 UTC

[single] Added documentation for the thread behavior

Added documentation for the thread behavior in various functions in single API
Also removed ERROR state as it was rendundant after #2360

Resolves #2185

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-05-07 17:58:16 UTC

[single] single API to process nnfw transparently once opened

single API should process all the frameworks transparently once the single handle has been opened.
The framework check has been performed when the handle was created and opened.
If the operation is not supported, that should be handled in the nnfw rather than in single API.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-05-07 17:32:24 UTC

[single] cache tensor filter class once at open

Existing implementation seeks the tensor filter class based on its type everytime a class is needed.
Rather, cache the tensor filter class increasing its ref at open, and unref at single api close.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-02-29 10:25:24 UTC

[filter] Support tensor filter framework V1 single API

Support tensor filter framework v1 callbacks for the single API

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-02-29 10:25:24 UTC

[filter] Support tensor filter framework V1

Support tensor filter framework v1 callbacks for the pipeline

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-02-21 08:59:31 UTC

[filter] Support get and set properties

Support get and set properties for GstTensorFilterFramework V1.
Updated GstTensorFilterFramework to include number of accelerators supported along with its list.

V2:
Added updating the properties
Corrected string comparison
replaced : else -> else if (GST_TF_FW_V1)
Updating properties which required caps renegotiation have been delayed
Added support for updating custom properties in eventHandler

V3:
Replaced asserts with critical logs
Added more comments for eventHandler return values

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-03-03 07:13:13 UTC

[meson] Add dependency of gtest

Some of the unittest depends on gtest
However their dependency is not checked leading to build failure with incorrect messages
Added dependency check of gtest before building those unittests

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-03-01 18:18:17 UTC

[tflite] optimize invoke critical path

Move unnecessary computations out of the invoke critical path to maximize invoke performance

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-02-20 03:33:37 UTC

[filter] Support probing for tensor filter subplugins version 1

Support probing for tensor filter subplugins framework of version 1
Various ToDos' have been added which will be supported in upcoming PRs

Related Issue: #2116

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-02-18 07:28:00 UTC

[subplugin/ext] Added unittest for remaining framework functions

Added support for basic unittests of destroyNotify, checkAvailability and allocateInInvoke

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-02-18 10:32:54 UTC

[unittest/build] Fix location of unittests binaries

Many unittest binaries are built and located in build/tests
All these binaries located in build/tests (not in recursive folders) are
run all at once with nnstreamer.spec by running all the files in build/tests (line 388)

However, some of these binaries are not supposed to run directly (they requires extra params),
and are properly run and tested with their shell scripts run with ssat (repo and reload unittests)

This PR moves the repo and reload unittest generated binaries into appropriate subfolders
to avoid running these binaries directly and avoid unnecessary warnings

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-02-11 10:24:21 UTC

[armnn/hw] Added supported acclerators

Added supported accelerators for armnn and added corresponding unittests with single API
Tested set accelerators on device with caffe model on the device

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-01-16 06:00:45 UTC

[ext] Remove class static variables without locking

As extensions are supposed to be thread-safe, using class static variables without locks is not stable.
Move class static variables to class private local object variables for python, tensorflow and caffe2.
Also update corresponding usages.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-02-03 10:18:26 UTC

[subplugin] Refactor subplugin interface

Refactor subplugin interface
Add new interface for the subplugins along with the old interface
Added version number to check if old subplugin interface is used or new one

Both the interfaces are added as a union,
This will allow for staggered changes rather a big change to be made at once

V2:
- Merged operation GET_INPUT_INFO and GET_OUTPUT_INFO into 1. Now atleast one of
SET_INPUT_INFO and GET_INOUT_INFO must be supported.
- Added framework versioning helper macros
- Older framework is now version 0 and newer one is version 1
- Renamed sendEvent to eventHandler
- private_data is a double pointer only in open/close in v1
- Invoke in v1 compared to invoke_NN in v0 where private_data is not a double pointer
- Moved init and fini functions in tensor_filter_cpp for readability

V3:
Update GstTensorFilterFramework function pointers set for all filters to pass all build

V4:
Added to description - eventHandler argument `void *data` is allowed to be NULL

V5:
Added tensor filter API versions for external subplugins to verify compatibility at compile time
Renamed GET_INOUT_INFO to GET_IN_OUT_INFO for readability

V6:
Added more operations to be supported with eventHandler which allow updating the properties
Added GstTensorFilterFrameworkEventData which allows arguments to be passed dependent on the event
GstTensorFilterProperties is passed to all the callbacks

V7:
Updated destroyNotify to take private_data as an input for V0, and corresponding changes in tensor_filter and subplugins
Added allocateInInvoke for V0

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-02-10 08:10:48 UTC

[single/coverity] Coverity fix

Added coverity fix of checking return value
In ml_single_set_info_in_handle: Value returned from a function is not checked for errors before being used (CWE-252)

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-02-07 09:03:29 UTC

[build/tizen] Identify build failure correctly

Identify the build failures correctly
Also disable the problematic unit-test of nnfw
This resolves #2081

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-02-07 09:06:24 UTC

[spec] spec file cleanup

Avoid running tests multiple times
More cleanup also added

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-02-07 08:21:00 UTC

[test] Sample failure test

Sample failure test

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-02-06 04:50:41 UTC

[tests] Clean up files after tests

Clean up files created during tests with ssat

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-01-29 08:06:04 UTC

[tensor_filter] Add unittest for data layout

Add unittests for added data layout property

V2:
Data layout can now be changed until the framework is opened
Once framwork is opened, updating the layouts will be supported later

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-01-31 12:01:21 UTC

[tensor_filter] Unused variable bug resolve

Resolve bug related to unused variable
If DBG is defined, this error is raised as the variable is only used
for check silent property when DBG is not defined

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-01-31 11:38:36 UTC

[Spec] Minor bug fix in spec

Add minor bug fix in spec

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-01-29 10:32:05 UTC

[single/caffe] Enable armnn in single-shot API

Enable support armnn for in single-shot API
Added test with caffe and tflite models

resolves #1817

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-01-08 05:51:10 UTC

[tensor] Support data layout for tensor

Data layout added for other/tensor
Three data layouts are supported
- NHWC - referred as channels last
- NCHW - referred as channels first
- none - unknown or doesnt matter
These data layouts can be different for each other/tensor in other/tensors
Also added their basic support in single API

V2:
appended prefix _NNS_LAYOUT to tensor_layout names

V3:
Added in layout in nnstreamer_examples to avoid getting stuck in some unittests

V4:
Layout will not be part of the gst element pad caps, and will not affect caps negotiation for now

V5:
None is moved at the top of enum for layout
Now layout always defaults to None

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-12-24 08:31:26 UTC

[test/ext] Add condition for function to exist

Verify that the function is available before running tests for that function
Added the above verification for getInputDimension and getOutputDimension
Further enhanced invoke test to run successfully if the getDimension exists

V2:
Added gst_tensors_info_free to catch memleak of tensor_info->name

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-12-31 10:58:46 UTC

[armnn] Add caffe support with armnn

Add caffe support with armnn
Add corresponding unit-test
Tested with CPU and GPU backend on the device

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-01-08 08:18:10 UTC

[nnfw] Verify input model filename

Added verification of input model filename
This is performed by parsing the metadata/MANIFEST file to be provided with the model
The first model in the list of models should be the model to be run by nnfw
json-glib is added as dependency to allow json parsing
Corresponding unittest is also added

Resolve: #1981

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-01-09 22:57:46 UTC

[pytorch] Enable setting accelerators

Enabled setting of accelerators
Enabled getting supported accelerators

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-01-09 12:19:03 UTC

[common] Added proper GstCaps framerate comparison

Added proper comparison of GstCaps framerate (1/2 == 2/4)
Added corresponding unittest cases

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-01-09 23:32:09 UTC

[tflite/bug] Set accelerator bug solve

Bug: Not setting accelerator ignored the use_nnapi configuration
This is resolved

Enable setting accelerator unittest non-tizen platforms as well

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-12-24 05:00:18 UTC

[python/ext] Added basic unittests for python

- Python initialize and finalize is done once for the module rather than doing it per object
Doing initialization per object causes errors when importing modules
- Replaced Assert with appropriate error handing for python extension of tensor filter
- Added basic unittests for python which covers the failure cases

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2020-01-03 04:49:33 UTC

[nnfw] Support setInputDimension with nnfw

With updated nnfw API, support setInputDimension with nnfw
Although the API is defined, its not yet implemeted interally
So, the unittest is added, but disabled

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-12-19 03:07:38 UTC

[single/hw] Check hw support for framework

Add implementation to check hw support for framework while checking
for its availability.
Added unittests for this functionality with tensorflow-lite

V2:
check hardware support performed while parsing for available hardwares
Split availability test cases into positive and negative

V3:
checkHWSupport->checkAvailability

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-12-23 10:29:41 UTC

[filter/ext] Added basic unittests for all filter extensions

Added basic unittests for all the the filter extensions
This creates failure cases of unittests for the interface defined with GstTensorFilterFramework
for most of the extensions, and performs basic unittests
Updated the unittest list in packaging files as well

Also added a script file to run all the test cases inside a folder

V2:
Split test cases into successful and negative ones

TODO: Support python extensions in this as well

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-12-31 11:37:25 UTC

[nnfw] Interpret quant8 asymm as uint8

Interpret quant8 asymm as uint8
Add unittest with object classification

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-12-31 07:39:24 UTC

[armnn] object classification unittest + bug solve

- Add object classification unittest using tflite quantized mobilenet
- Add orange.png raw format image orange.raw
- As armnn treats uint8 datatype of tflite as quant asymm8 (arc/armnnTfLiteParser/TfLiteParser.cpp:310),
quant asymm8 is interpreted as uint8 in nnstreamer
- Updated current default HW backend to CpuAcc (based on neon) as CpuRef takes over
5 min to complete the mobilenet unittest

Bug solve:
The dimension of data received by armnn are inversed to match the desired order

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-12-17 04:51:24 UTC

[armnn] Added unittests for armnn

Added unittests for armnn tensor filter extension

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-11-14 08:25:06 UTC

[armnn] Added tensor filter extension of armnn

Added tensor filter extension of armnn

V2: Updated the static library install paths for armnn and nnfw subplugins
V3: Updated error handling
V4: Remove Wno-sign-compare

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-12-17 06:02:42 UTC

[plugin/hw] Add more HW accelerator and update implementation

Add TPU as a backend accelerator property option in list of hardware
accelerators
Move default accelerator to avoid changing default accelerator enum value
everytime a new backend device is added

V2:
- Updated the regex to be formed at run-time using a list of supported
accelerators.
- Accelerators are also stored as GEnumValues to do conversion from
string to int value and vice versa easily
- More accelerators are added in heirarchical order. srcn and neon
moved to npu and cpu heirarchy correspondingly.
- Added more accelerators to be supported in future.

V3:
Update backend accelerator in tizen c-api

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-12-27 13:50:12 UTC

[nnfw] Reverse the order of dimension

Reverse the order of dimensions with nnstreamer as
nnfw stores dimension in accordance with tflite

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-12-23 08:24:55 UTC

[filter/ext] Replace asserts with error handling

Replace asserts with appropriate error handling in tensor filter extensions
This will allows test cases for failure scenarios and handle erronous inputs
via single-shot API gracefully with error return than shutting down with assert failure

V2:
Applied for custom filters as well

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-12-19 04:19:52 UTC

[tflite] Backend unittest should always pass

Tensorflow-lite unittest which tests for appropriate backend parsing and testing
should always pass.
Removed accelerator input validity check at tensor_filter.c as it is checked in
each filter again.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-12-18 05:47:14 UTC

[capi/test] Split capi into multiple files

capi test file is growing, and needs splitting

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-12-16 11:22:21 UTC

[nnfw] Get dimension bug reproduction code

This code reproduces bug with nnfw_input_tensorinfo and
nnfw_output_tensorinfo. The input and output dimensions
obtained from nnfw are rank=1 and dim=1, which is different
from the true dimension of the model.
This PR allows reproduction of the bug.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-12-16 02:19:49 UTC

[spec] nnfw now packaged only for tizen

nnfw package is now created and corresponding tests performed
only for tizen in the spec file.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-11-08 07:30:51 UTC

[nnfw/single/test] Add tests for nnfw with single API

Add invoke test for nnfw with single API

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-11-21 01:21:26 UTC

[IIO/test/coverity] Resolve coverity issues

Added more checks coherent with in #1859 and #1863 to another location

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-11-21 01:44:21 UTC

[tests/coverity] Fix coverity issues related to gtest main

Add try/catch for exceptions thrown by function calls in main bosy when using gtest

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-11-20 12:54:44 UTC

[Docs] Updated the mirrors for debian build

Updated the mirrors for debian build in the getting-started docs
This now complies with the list of mirrors used by launchpad of nnstreamer

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-10-26 06:53:53 UTC

[Android] Flexible input dimension testcases

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-10-24 05:15:04 UTC

[single] Fix possible svace issue later

Fix possible svace issue later
https://github.com/nnsuite/nnstreamer/pull/1787#discussion_r337823705

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-10-22 04:27:04 UTC

[pytorch] Do first run with try/catch

First run after configuring the tensor filter with pytorch framework
is now run with a try/catch.

Check #1809 for more details.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-10-21 10:28:44 UTC

[single] More inline functions for flexible dimensions

Added extra inline funcitons to provide easier API for dyanmic input dimensions

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-10-18 03:41:53 UTC

[TF-Lite] Added setInputDimension

Added implementation for setInputDimension
This can allow support for flexible input dimensions
The support is limited to only changing dimensions
Data type for the input or the number of inputs cannot be changed with current TF-Lite API
The rank for the input is allowed to be changed if the model allows
The input dimension can return error if the model does not allow dimension to be changed

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-10-21 03:00:18 UTC

[single] simplify get_tensors_info in single API

Added simplification for get_tensors_info in single API implementation
input and output tensors info is stored in single API
This patch updates to use that information than extracting the info again from element

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-10-17 10:32:16 UTC

[single] Implementation of flexible input dimension

Added implementation of flexible input dimension at the API level
Handling flexible input dimension at the tensor filter extension
scope to be done.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-10-21 10:28:44 UTC

[single] More inline functions for flexible dimensions

Added extra inline funcitons to provide easier API for dyanmic input dimensions

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-10-16 04:25:39 UTC

[tests] Install models/data with tests

Install models/data with tests correctly when needed
Install tizen-capi-unittest with install-test option

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-10-16 04:08:53 UTC

[tests] Locate all models and data together

Locate all the models and data together for easier management

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-10-10 12:48:59 UTC

[tizen/nnfw] Add get_input/output_dimensions

Added implementation for get_input/output_dimensions functions

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-10-07 04:27:19 UTC

[single] Minor updates for modularity

Added minor updates in C-API single shot implementation for improved modularity

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-10-02 07:37:18 UTC

[android/build] Update build script

Update build script to be directly used by CI for building

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-10-02 04:54:23 UTC

[single] Increase default timeout

Increase the default timeout to pass the test case on a slow machine/not fully
optimized tensorflow-lite. Also, as the running time for tf-lite test case in
aarch64 is over 10x compared to armv7l, higher defailt timeout is better.

In unit-test, increased the corresponding default timeout as well.
Further, when running many test cases in parallel, default timeout is scaled
with number of parallel runs.

Solves #1717

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-09-26 03:03:23 UTC

[svace/iio] security fixes related to sscanf

1. Fix security issue related with sscanf. Use g_ascii_str instead
Issue: WID:7938408 Use of vulnerable function 'sscanf' at tensor_src_iio.c:690. This function is unsafe.
Issue: WID:7938409 Use of vulnerable function 'sscanf' at tensor_src_iio.c:723. This function is unsafe.
2. Add comments for switch cases where breaks have been skipped
Issue: Missing break in the switch case

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-09-20 07:22:27 UTC

[single-new] Memory leak bug fix in error cases

Added memory leak bug fix in error cases running

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-09-20 10:13:16 UTC

[single-new] Remove dependency over pthread

Remove the dependency over pthread
Use glib functionalities instead

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>

Author: Parichay Kapoor
Author Date: 2019-09-19 03:56:34 UTC

[single] Thread safety bug fix

Added bug fix for thread safe usage
Single shot API use single handle lock for making all its API call thread safe
However, the call to close the API destroys the handle itself making the
call to close not thread safe
Added a global lock to make closing of handles thread safe

Further, all the single handle locks are acquired inside the global lock
to ensure that single handle is not closed while the magic has been verified
and the local lock is not acquired

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>