public class AST2BOpFilters extends AST2BOpBase
AST2BOpBase.Annotations
Modifier | Constructor and Description |
---|---|
protected |
AST2BOpFilters() |
Modifier and Type | Method and Description |
---|---|
protected static PipelineOp |
addChunkedMaterializationStep(PipelineOp left,
Set<IVariable<IV>> vars,
boolean materializeInlineIvs,
Long cutoffLimit,
Properties queryHints,
AST2BOpContext ctx)
Use a pipeline operator which uses the chunked materialization pattern
for solution sets.
|
protected static PipelineOp |
addMaterializationSteps1(PipelineOp left,
int rightId,
IValueExpression<IV> ve,
Set<IVariable<IV>> vars,
Properties queryHints,
AST2BOpContext ctx)
If the value expression that needs the materialized variables can run
without a
NotMaterializedException then just route to the
rightId (around the rest of the materialization pipeline steps). |
protected static PipelineOp |
addMaterializationSteps2(PipelineOp left,
int rightId,
Set<IVariable<IV>> vars,
Properties queryHints,
AST2BOpContext ctx)
Adds a series of materialization steps to materialize terms needed
downstream.
|
protected static PipelineOp |
addMaterializationSteps3(PipelineOp left,
Set<IVariable<?>> doneSet,
Map<IConstraint,Set<IVariable<IV>>> needsMaterialization,
Properties queryHints,
AST2BOpContext ctx)
For each filter which requires materialization steps, add the
materializations steps to the pipeline and then add the filter to the
pipeline.
|
protected static PipelineOp |
addNonConditionalMaterializationSteps(PipelineOp left,
Set<IVariable<?>> doneSet,
Map<IConstraint,Set<IVariable<IV>>> needsMaterialization,
Long cutoffLimit,
Properties queryHints,
AST2BOpContext ctx)
The RTO requires that we do not reorder solutions.
|
protected static IConstraint[] |
getJoinConstraints(Collection<IConstraint> constraints,
Map<IConstraint,Set<IVariable<IV>>> needsMaterialization)
Partition the constraints for a join into those which can (or might) be
able to run attached to that join and those which must (or might) need to
materialize some variables before they can be evaluated.
|
protected static List<IConstraint> |
getJoinConstraints(IJoinNode joinNode)
Convert the attached join filters into a list of
IConstraint s. |
protected static IConstraint[] |
getJoinConstraints2(Collection<IConstraint> constraints,
Map<IConstraint,Set<IVariable<IV>>> needsMaterialization,
boolean conditionalRouting)
Partition the constraints for a join into those which can (or might) be
able to run attached to that join and those which must (or might) need to
materialize some variables before they can be evaluated.
|
applyQueryHints, applyQueryHints, leftOrEmpty
protected static PipelineOp addMaterializationSteps1(PipelineOp left, int rightId, IValueExpression<IV> ve, Set<IVariable<IV>> vars, Properties queryHints, AST2BOpContext ctx)
NotMaterializedException
then just route to the
rightId (around the rest of the materialization pipeline steps).
This happens in the case of a value expression that only "sometimes"
needs materialized values, but not always (i.e. materialization
requirement depends on the data flowing through). A good example of this
is CompareBOp
, which can sometimes work on internal values and
sometimes can't.
To materialize the variable ?term
, the pipeline looks as
follows:
left -> ConditionalRoutingOp1 (condition=!IsMaterialized(?term), alt=rightId) -> ConditionalRoutingOp2 (condition=IsInline(?term), alt=PipelineJoin) -> InlineMaterializeOp (predicate=LexPredicate(?term), sink=rightId) -> PipelineJoin (predicate=LexPredicate(?term)) -> rightId
left
- The left (upstream) operator that immediately proceeds the
materialization steps.rightId
- The id for the right (downstream) operator that immediately
follows the materialization steps. This needs to be
pre-reserved by the caller.ve
- The IValueExpression
for the SPARQLConstraint
.varsToMaterialize
- The variables to be materialize.queryHints
- The query hints to be applied from the dominating operator
context.ctx
- The evaluation context.TryBeforeMaterializationConstraint
,
AST2BOpUtility#addMaterializationSteps1(PipelineOp, int,
IValueExpression, Collection, AST2BOpContext)
protected static PipelineOp addMaterializationSteps2(PipelineOp left, int rightId, Set<IVariable<IV>> vars, Properties queryHints, AST2BOpContext ctx)
?term
, the pipeline
looks as follows:
(A) leftId : The upstream operator (B) condId1 : if !materialized then condId2 (fall through) else rightId (C) condId2 : if inline then inlineMatId (fall through) else lexJoinId (D) inlineMatId : InlineMaterializeOp; goto rightId. (E) lexJoinId : LexJoin (fall through). (F) rightId : the downstream operator.
left -> ConditionalRoutingOp1 (condition=!IsMaterialized(?term), alt=rightId) -> ConditionalRoutingOp2 (condition=IsInline(?term), alt=PipelineJoin) -> InlineMaterializeOp (predicate=LexPredicate(?term), sink=rightId) -> PipelineJoin (predicate=LexPredicate(?term)) -> rightId
left
- The left (upstream) operator that immediately proceeds the
materialization steps.rightId
- The id of the right (downstream) operator that immediately
follows the materialization steps.vars
- The terms to materializequeryHints
- The query hints from the dominating AST node.ctx
- The evaluation context.TryBeforeMaterializationConstraint
protected static PipelineOp addChunkedMaterializationStep(PipelineOp left, Set<IVariable<IV>> vars, boolean materializeInlineIvs, Long cutoffLimit, Properties queryHints, AST2BOpContext ctx)
Note: The RTO uses this method since it does not use conditional routing and does not reorder the solutions.
left
- The left (upstream) operator that immediately proceeds the
materialization steps.vars
- The terms to materialize.materializeInlineIvs
- When true
, inline IVs are also materialized.queryHints
- The query hints from the dominating AST node.ctx
- The evaluation context.ChunkedMaterializationOp
protected static PipelineOp addMaterializationSteps3(PipelineOp left, Set<IVariable<?>> doneSet, Map<IConstraint,Set<IVariable<IV>>> needsMaterialization, Properties queryHints, AST2BOpContext ctx)
left
- doneSet
- The set of variables already known to be materialized. This is
populated as a side-effect with any variables that will be
materialized by the materialization steps added by this
method.needsMaterialization
- A map of constraints and their variable materialization
requirements.queryHints
- Query hints from the dominating AST node.ctx
- The evaluation context.
TODO This treats each filter in turn rather than handling all
variable materializations for all filters at once. Is this
deliberate? If so, maybe we should pay attention to the order
of the filters. I.e., those constraints should be ordered
based on an expectation that they can reduce the total work by
first eliminating solutions with less materialization effort
(run constraints without materialization requirements before
those with materialization requirements, run constraints that
are more selective before others, etc.).protected static PipelineOp addNonConditionalMaterializationSteps(PipelineOp left, Set<IVariable<?>> doneSet, Map<IConstraint,Set<IVariable<IV>>> needsMaterialization, Long cutoffLimit, Properties queryHints, AST2BOpContext ctx)
1. We can not attach a filter with SOMETIMES requirements to a JOIN and
wrap it with a TryBeforeMaterializationConstraint
since this
requires a ConditionalRoutingOp
with an altSink and that will
reorder solutions.
2. We can not use a pattern which involves an InlineMaterializeOp
followed by a ConditionalRoutingOp
with an altSink followed by a
PipelineJoin
against the lexicon. This also reorders the
solutions (primarily because of the ConditionalRoutingOp
since we
can force the PipelineJoin
to not reorder solutions).
The code below uses the ChunkedMaterializationOp
. This does not
reorder the solutions. It can also materialize inline IVs giving us a
single operator that prepare the solutions for filter evaluation.
protected static IConstraint[] getJoinConstraints(Collection<IConstraint> constraints, Map<IConstraint,Set<IVariable<IV>>> needsMaterialization)
TryBeforeMaterializationConstraint
which will ignore errors
caused by unmaterialized values and in the needsMaterialization
map. This allows such constraints to run attached to the join iff that is
possible and otherwise to be evaluated as soon as the materialization
pipeline can satisify their materialization requirements.constraints
- The constraints (if any) for the join (optional). This
collection is NOT modified.needsMaterialization
- A map providing for each constraint the set of variables which
must be materialized before that constraint can be evaluated.
This map is populated as a side-effect. It will be empty iff
there are no constraints that might or must require variable
materialization.null
iff there are no constraints that can
be attached to the join.protected static IConstraint[] getJoinConstraints2(Collection<IConstraint> constraints, Map<IConstraint,Set<IVariable<IV>>> needsMaterialization, boolean conditionalRouting)
TryBeforeMaterializationConstraint
which will ignore errors
caused by unmaterialized values and in the needsMaterialization
map. This allows such constraints to run attached to the join iff that is
possible and otherwise to be evaluated as soon as the materialization
pipeline can satisify their materialization requirements.
Note: The RTO requires that solutions are not reordered during cutoff JOIN evaluation in order to obtain accurate estimates of the cardinality of a join based on a known number of solutions in producing a known number of solutions out. Conditional materialization can cause solutions to be reordered since some solutions may pass the constraint attached to the join and be routed along one path in the query plan while other solutions may pass the join but fail the constraint due to a materialization requirement and are routed along another path. Thus it is necessary to disable conditional routing for cutoff JOIN evaluation.
constraints
- The constraints (if any) for the join (optional). This
collection is NOT modified.needsMaterialization
- A map providing for each constraint the set of variables which
either might or must be materialized before that constraint
can be evaluated. This map is populated as a side-effect. It
will be empty iff there are no constraints that might or must
require variable materialization.conditionalRouting
- When true
, constraints that
INeedsMaterialization.Requirement.SOMETIMES
are able to run attached are
wrapped by a TryBeforeMaterializationConstraint
and
appear both attached to the JOIN and will also run after the
JOIN using a ConditionalRoutingOp
to route solutions
that fail the attached contraint through a materialization
pipeline and then into a 2nd copy of the constraint once the
variable(s) are known to be materialized.false
, only constraints that
INeedsMaterialization.Requirement.NEVER
require materialization will be
attached to the JOIN. All other constraints will use
non-conditional materialization.null
iff there are no constraints that can
be attached to the join.protected static List<IConstraint> getJoinConstraints(IJoinNode joinNode)
IConstraint
s.Copyright © 2006–2019 SYSTAP, LLC DBA Blazegraph. All rights reserved.