systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthias Boehm <>
Subject Re: Loss of dimensionality info in transient reads
Date Tue, 18 Apr 2017 23:21:53 GMT
In general, there are a couple of scenarios which make size propagation
challenging. This includes:

* Complex function call patterns (where functions are potentially called
with different sizes)
* External user-defined functions
* Data-dependent operators (e.g., table, aggregate, removeEmtpy);
* Computed size expressions
* Changing dimensions or sparsity across iterations

Core components that deal with the related issue of size propagation are
our inter-/intra-procedural analysis and dynamic recompilation. So if you
encountered scenarios that we did not handle very well, please share and we
can see how to extend these components.

Note, however, that not all size information is exposed in explain. The
explain output only shows exact dimension or sparsity information. In some
cases, we cannot infer the exact sizes but worst-case information which is
used for example for memory estimates. These worst-case estimate require
special treatment because size expressions such as B=rand(rows=N-nrow(A),
cols=1) can turn a safe over-estimate of A into a dangerous under-estimate
of B.


On Tue, Apr 18, 2017 at 12:21 PM, <> wrote:

> In which scenarios would a transient read lose the row/col dimensionality
> information?
> --
> Mike Dusenberry
> GitHub:
> LinkedIn:
> Sent from my iPhone.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message