Changes between Version 19 and Version 20 of DataParallel/Replicate


Ignore:
Timestamp:
Aug 7, 2011 2:29:07 PM (3 years ago)
Author:
chak
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • DataParallel/Replicate

    v19 v20  
    9393 TODO: rl's example 
    9494 
    95 === Splitting and combining === 
     95=== Splitting and combining (for lifted conditions) === 
    9696 
    9797Due to lifted conditionals (or, generally, case constructs), arrays with repeated segments may be split (packed) and combined.  Arrays with repeated segments can be split (or packed) by including a repeated segment in the result exactly if one of its repetitions is included.  This can be determined by disjunctively combining all flags for one sequence of repeated segments. 
    9898 
    9999Arrays with repeated segments can not always be combined without duplicating the data corresponding to repeated segments (after all, a disjoint segment may be inserted into a sequence of repeated segments).  For simplicity, we may want to expand all repeated segments in `combinePA`.  (It seems that this should not lead to unexpected blow-ups as the repeated data now is part of a functions result and should be accounted for in its complexity estimate.) 
     100 
     101=== Splitting and joining (for distribution across threads === 
     102 
     103Our idea is to continue to split the '''data''' array evenly across threads.  That may spread out the segments descriptor (length and starts arrays) very unevenly as repeated segments have an entry for each repetition in the segmentation information, but not in the data. 
     104 
     105TODO: Roman believes splitting arrays with repeated segments is a problem.  To me it doesn't seem to be that much of a problem.  (Keep in mind that we only need a restricted number of operations on arrays with repeated segments — all their consumers are homomorphisms as discussed in Plan B. 
    100106 
    101107=== Multiple levels of nesting (unconcat and concat) ===