Hi community,

I want to update an old project using kinect1 sensor with a KinectV2.
Since KinectV2 only works in windows, I am doing everything on Win 10.

Right now I receive the sensor's data using this small OSC utility https://github.com/microcosm/KinectV2-OSC

The data looks like this:

print: /bodies/72057594037928550/joints/Head 0.321555 -0.0569061 0.743229 Inferred

The ID is changing everytime when a user enter the field of view / or more than one user are present in the field of view.

I need only one user ID, the clothest. So I need to make a function that route the body ID and keep it as long as his head posZ is smaller than other users ID head posZ.

Here is an attempt, I did a little vanilla Kinect data simulator. If somebody can take a look and give any hint it would be very nice

Cheers

Kinect-V2-skeleton-route.zip

Anybody on this?

Right now I made a simple rule to route the current ID as long as the body is present.. but It's just a simple patch that I need to improve.

I still need to parse a list regarding the z position of the head...

@60hz Sorry, but all I have for you are questions . Do you know beforehand the complete set of possible IDs? I'm assuming from your description and example that each ID may go in and out of range, so when you say you want the closest one, do you mean the closest one that's still in range? And I gather from your example that an ID is in-range when you've received an OSC message from it within the last 100 ms, is that right?

@60hz Maybe you can just put all IDs in a [text] and use [text search] to find the lowest ID?

@ingox That's probably the simplest, but Id's need to be removed as they go out of bounds, or have been "untracked" for a small period of time.
Maybe time stamp the data and give it a short life in the text file.
Medical research also has some thoughts on what Kinect data should be treated as valid.
For example "Inferred" should probably be dropped entirely as it could only be valid for the smallest head z if another ID put their hands around that head from behind.
David.

@jameslo the ID's are generated randomly each time a new user come in the field of view. The second example is just a quick hack to make something "quickly work" but the real attempt is in the patch from the zip file (with a simple Kinect user data simulator)

@ingox the thing is I don't need to find the lowest ID, but to route the list of the ID which contain the smallest Z head position in the list. For example if I have 2 ID incoming:

print: /bodies/000000001/joints/Head 0.321555 -0.0569061 0.2 Inferred
print: /bodies/000000002/joints/Head 0.321555 -0.0569061 0.1 Inferred

then, I need to route only the list which have the smallest head z position:

print: /bodies/000000002/joints/Head 0.321555 -0.0569061 0.1 Inferred

@whale-av I guess you talk about occlusion here, the last list element is trackingState (Tracked, NotTracked or Inferred), I guess I need to think about it also but it's another step ahead.

@60hz The "near" parameter for [text search] combined with the empty [route] filter from your first patch will allow an increasing z for the smallest head.
That allows for a head Z to start increasing in value while still being the smallest.
You don't actually need to find the lowest value. You just want to find the line with the Z value closest to zero.

@whale-av Yes, exactly!

@60hz Try this.........kin5.zip
I think all the problems are resolved......... and the store size (number of recent IDs stored) can be adjusted dynamically.
With simple test.... kin6.zip
David.

Wow thanks @whale-av, that looks like a clever way to do!

But If the user who have the lower head leave the field, the remaining user who have the lowest are not present since I reset all (or clear "smallest" text define) isn't it?
The kinectv2 could see up to 6 user, so if many people are in traked in the room how to do?

@60hz I don't know the exact meaning of the trailing data...... Inferred and tracked make sense.
But if "untracked" means the user has left the space then that could be used to delete the line for that user.
It could also be done with an x or y range test or probably just x...... set a window size and window edge width and as their head falls within the edge boundary delete them.
Probably safest to have y as well in case they crawl out of view of jump out of the edge

If you only have 6 bodies the store is not going to grow. Any store setting >=6 will be fine.
Maybe setting the store size at 5 would help as the store would cycle the users while there are 6 and drop any user that leaves.
But then when there are only 5 it would stop cycling and fail to detect another one leaving.
I am not keen on that solution. The store could be reduced again after a set time but that is unsatisfactory and would introduce errors anyway.
Using reset will (very) temporarily introduce errors too..... and an admin needs to be awake for that.
It will rebuild the store very fast, but there will be some incorrect data points until it is built (all current users are added).

What is the x and the y range for the data?
I am thinking..... when a head is in the red area it is deleted from the store....... when it enters the white area it is added to the store..... sort of like this.......

.... so eg for an x range 0 to 1.........
if x <= 0.05 or x >= 0.95 delete else add
David.

@whale-av said:

Inferred and tracked

it give information about a specific part of the body so the program can make that part disappear in order to keep a less jittery body on screen I guess (that's the way I will do next).

Using reset will (very) temporarily introduce errors too.....

Yes, I think also that it might be problematic.

I am thinking..... when a head is in the red area it is deleted from the store.......

Yes, that way looks even more suitable!

@60hz So what does "Untracked" mean? If it means "left the scene" we can use that.
How to treat "Inferred"........ ignore it?
And the x and y data ranges for setting the borders if "Untracked" means something else (and then how to treat it if it means something else..... ignore it as well?)..
David.

@whale-av
Actually I think that a user is still detected even if some part (like the head) leave the file of view... so relying on it being "untracked" means there will still be a working ID in the field of view for other parts of the body.

Unfortunately With KinectOSC, there is no global message regarding the tracking of a user, that's why I used the delay between information before for my test.

@60hz Looking at this...... https://silo.tips/download/real-time-person-tracking-and-identification-using-the-kinect-sensor-major-quali it looks like a decision has to be made about the number of times a head is untracked before it is removed ( data reliability).
So boundaries are a good way to go, and with a width setting so as to adjust for optimum in a live situation ( catching a head exiting at speed).
I think it is worth putting a switch to decide if Inferred should be ignored or not, and trying it in a live situation.
"Untracked" can then be ignored.
Once you know x and y ranges (0-1?) it will be easy.
David.

Hi, all. I'm curious: will the Xbox Kinect V2 work with this approach or is this the same deal as before where I need dedicated Windows hardware to do some tracking? I've been totally out of the loop on this stuff for years so I'm curious if anything has changed in that regard.

@whale-av
I am finally back on it and made it work finding the clothest X center position of the head and a reset after the last skeleton exit... I don't now how it behaves with many people yet...

@ricky
As far as I can tell, we still need a terrifying windows system in order to get skeleton tracking with Kinect v2.