Partitioning of distance matrices
Posted: Fri Sep 20, 2024 7:58 am
Hi there,
this week several players asked me about almost the same requirement - here's the story, based on xServer2:
But: The internal calculation of distance matrices through the dima engine is not limited to such dimensions. As you can see in Dima calculation times xDima2 (HPR) we can compute much bigger matrices with up to (100'000)² relations. So the challenge is
Imagine a (3'000)² matrix created on the server via a single, regular com.ptvgroup.xserver.xdima.CreateDistanceMatrixRequest (based on HPR)
this week several players asked me about almost the same requirement - here's the story, based on xServer2:
So this stratement is driven by the usage of the distance matrix content not within the PTV servers but on the client side. So the user is familiar with the method com.ptvgroup.xserver.xdima.CreateAndGetDistanceMatrixRequest which offersWe have encountered a situation where we need a distance (and ETA) matrix exceeding the current limit of 250,000 relations. According to our estimates, we expect to work with matrices of approximately 3'000x3'000 in size.
I have learned that there may be a solution to increase this limit. Could you share your opinion on the feasibility of increasing the matrix size limit? Additionally, do you have a good walkthrough or guidelines on how we could achieve this?
- to create the dima content
- to download the content on clientside
But: The internal calculation of distance matrices through the dima engine is not limited to such dimensions. As you can see in Dima calculation times xDima2 (HPR) we can compute much bigger matrices with up to (100'000)² relations. So the challenge is
Well : nobody said, that you have to consume the data in a single transaction. Just cut that thing into pieces!If the huge data is produced and stored on the server: how to transfer it to the client without being restricted by the 250'000 limit?
Imagine a (3'000)² matrix created on the server via a single, regular com.ptvgroup.xserver.xdima.CreateDistanceMatrixRequest (based on HPR)
- get 3'000 slices of the shape [1]:[3000] (3'000 relations each) - through com.ptvgroup.xserver.xdima.GetDistanceMatrixByLocationsRequest
- Or even bigger substructures such as 100 calls of shape [30]x[3000] (90'000 relations each)
- Accessing the distance matrix contents using the binary encoded version is mandatory!