org.apache.spark

Class ComplexFutureAction<T>

java.lang.Object

org.apache.spark.ComplexFutureAction<T>

All Implemented Interfaces:

FutureAction<T>, scala.concurrent.Awaitable<T>, scala.concurrent.Future<T>

public class ComplexFutureAction<T>
extends java.lang.Object
implements FutureAction<T>

A FutureAction for actions that could trigger multiple Spark jobs. Examples include take, takeSample. Cancellation works by setting the cancelled flag to true and interrupting the action thread if it is being blocked by a job.

Nested Class Summary

Nested classes/interfaces inherited from interface scala.concurrent.Future

scala.concurrent.Future.InternalCallbackExecutor$

Constructor Summary

Constructors

Constructor and Description
ComplexFutureAction()

Method Summary

Methods

Modifier and Type	Method and Description
void	cancel() Cancels the execution of this action.
boolean	isCancelled() Returns whether the action has been cancelled.
boolean	isCompleted() Returns whether the action has already been completed with a value or an exception.
scala.collection.Seq<java.lang.Object>	jobIds() Returns the job IDs run by the underlying async operation.
<U> void	onComplete(scala.Function1<scala.util.Try<T>,U> func, scala.concurrent.ExecutionContext executor) When this action is completed, either through an exception, or a value, applies the provided function.
ComplexFutureAction<T>	ready(scala.concurrent.duration.Duration atMost, scala.concurrent.CanAwait permit) Blocks until this action completes.
T	result(scala.concurrent.duration.Duration atMost, scala.concurrent.CanAwait permit) Awaits and returns the result (of type T) of this action.
ComplexFutureAction<T>	run(scala.Function0<T> func, scala.concurrent.ExecutionContext executor) Executes some action enclosed in the closure.
<T,U,R> void	runJob(RDD<T> rdd, scala.Function1<scala.collection.Iterator<T>,U> processPartition, scala.collection.Seq<java.lang.Object> partitions, scala.Function2<java.lang.Object,U,scala.runtime.BoxedUnit> resultHandler, scala.Function0<R> resultFunc) Runs a Spark job.
scala.Option<scala.util.Try<T>>	value() The value of this Future.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.FutureAction

get

Methods inherited from interface scala.concurrent.Future

andThen, collect, failed, fallbackTo, filter, flatMap, foreach, map, mapTo, onFailure, onSuccess, recover, recoverWith, transform, withFilter, zip

Constructor Detail

ComplexFutureAction

public ComplexFutureAction()

Method Detail

cancel

public void cancel()

Description copied from interface: FutureAction

Cancels the execution of this action.

Specified by:

cancel in interface FutureAction<T>

run

public ComplexFutureAction<T> run(scala.Function0<T> func,
                         scala.concurrent.ExecutionContext executor)

Executes some action enclosed in the closure. To properly enable cancellation, the closure should use runJob implementation in this promise. See takeAsync for example.

Parameters:

func - (undocumented)

executor - (undocumented)

Returns:

(undocumented)

runJob

public <T,U,R> void runJob(RDD<T> rdd,
                  scala.Function1<scala.collection.Iterator<T>,U> processPartition,
                  scala.collection.Seq<java.lang.Object> partitions,
                  scala.Function2<java.lang.Object,U,scala.runtime.BoxedUnit> resultHandler,
                  scala.Function0<R> resultFunc)

Runs a Spark job. This is a wrapper around the same functionality provided by SparkContext to enable cancellation.

Parameters:

rdd - (undocumented)

processPartition - (undocumented)

partitions - (undocumented)

resultHandler - (undocumented)

resultFunc - (undocumented)

isCancelled

public boolean isCancelled()

Description copied from interface: FutureAction

Returns whether the action has been cancelled.

Specified by:

isCancelled in interface FutureAction<T>

Returns:

(undocumented)

ready

public ComplexFutureAction<T> ready(scala.concurrent.duration.Duration atMost,
                           scala.concurrent.CanAwait permit)

Description copied from interface: FutureAction

Blocks until this action completes.

Specified by:

ready in interface FutureAction<T>

Specified by:

ready in interface scala.concurrent.Awaitable<T>

Parameters:

atMost - maximum wait time, which may be negative (no waiting is done), Duration.Inf for unbounded waiting, or a finite positive duration

permit - (undocumented)

Returns:

this FutureAction

result

public T result(scala.concurrent.duration.Duration atMost,
       scala.concurrent.CanAwait permit)

Description copied from interface: FutureAction

Awaits and returns the result (of type T) of this action.

Specified by:

result in interface FutureAction<T>

Specified by:

result in interface scala.concurrent.Awaitable<T>

Parameters:

atMost - maximum wait time, which may be negative (no waiting is done), Duration.Inf for unbounded waiting, or a finite positive duration

permit - (undocumented)

Returns:

the result value if the action is completed within the specific maximum wait time

onComplete

public <U> void onComplete(scala.Function1<scala.util.Try<T>,U> func,
                  scala.concurrent.ExecutionContext executor)

Description copied from interface: FutureAction

When this action is completed, either through an exception, or a value, applies the provided function.

Specified by:

onComplete in interface FutureAction<T>

Specified by:

onComplete in interface scala.concurrent.Future<T>

Parameters:

func - (undocumented)

executor - (undocumented)

isCompleted

public boolean isCompleted()

Description copied from interface: FutureAction

Returns whether the action has already been completed with a value or an exception.

Specified by:

isCompleted in interface FutureAction<T>

Specified by:

isCompleted in interface scala.concurrent.Future<T>

Returns:

(undocumented)

value

public scala.Option<scala.util.Try<T>> value()

Description copied from interface: FutureAction

The value of this Future.

If the future is not completed the returned value will be None. If the future is completed the value will be Some(Success(t)) if it contains a valid result, or Some(Failure(error)) if it contains an exception.

Specified by:

value in interface FutureAction<T>

Specified by:

value in interface scala.concurrent.Future<T>

Returns:

(undocumented)

jobIds

public scala.collection.Seq<java.lang.Object> jobIds()

Description copied from interface: FutureAction

Returns the job IDs run by the underlying async operation.

This returns the current snapshot of the job list. Certain operations may run multiple jobs, so multiple calls to this method may return different lists.

Specified by:

jobIds in interface FutureAction<T>

Returns:

(undocumented)