TransactionManager (Apache Tephra 0.13.0-incubating API)

java.lang.Object
+
- com.google.common.util.concurrent.AbstractService
- +
  - org.apache.tephra.TransactionManager
  +
+

+
+
All Implemented Interfaces:
+
com.google.common.util.concurrent.Service
+
+
+
+
```
public class TransactionManager
+extends com.google.common.util.concurrent.AbstractService
```
+
This is the central place to manage all active transactions in the system. + + A transaction consists of +
- A write pointer: This is the version used for all writes of that transaction.
- A read pointer: All reads under the transaction use this as an upper bound for the version.
- A set of excluded versions: These are the write versions of other transactions that must be excluded from + reads, because those transactions are still in progress, or they failed but couldn't be properly rolled back.
+ To use the transaction system, a client must follow this sequence of steps: +
1. Request a new transaction.
2. Use the transaction to read and write datasets. Datasets are encouraged to cache the writes of the + transaction in memory, to reduce the cost of rollback in case the transaction fails.
3. Check whether the transaction has conflicts. For this, the set of change keys are submitted via canCommit(), + and the transaction manager verifies that none of these keys are in conflict with other transactions that + committed since the start of this transaction.
4. If the transaction has conflicts: +
  1. Roll back the changes in every dataset that was changed. This can happen in-memory if the + changes were cached.
  2. Abort the transaction to remove it from the active transactions.
  +
5. If the transaction has no conflicts:
+ Transactions may be short or long-running. A short transaction is started with a timeout, and if it is not + committed before that timeout, it is invalidated and excluded from future reads. A long-running transaction has + no timeout and will remain active until it is committed or aborted. Long transactions are typically used in + map/reduce jobs and can produce enormous amounts of changes. Therefore, long transactions do not participate in + conflict detection (they would almost always have conflicts). We also assume that the changes of long transactions + are not tracked, and therefore cannot be rolled back. Hence, when a long transaction is aborted, it remains in the + list of excluded transactions to make its writes invisible.
+

+ +

Nested Class Summary

+ + + + + + + + + + + + + + + + + + +

Nested Classes
Modifier and Type	Class and Description
`static class`	`TransactionManager.ChangeSet` + Represents a set of changes from a client. +
`static class`	`TransactionManager.InProgressTx` + Represents some of the info on in-progress tx +
`static class`	`TransactionManager.InProgressType` + Type of in-progress transaction. +

+ + +
Nested classes/interfaces inherited from interface com.google.common.util.concurrent.Service
+com.google.common.util.concurrent.Service.Listener, com.google.common.util.concurrent.Service.State

+ +

+ + +

Constructor Summary

+ + + + + + + + + + + +

Constructors
Constructor and Description
`TransactionManager(org.apache.hadoop.conf.Configuration config)`
`TransactionManager(org.apache.hadoop.conf.Configuration conf, + TransactionStateStorage persistor, + MetricsCollector txMetricsCollector)`

+ +

+ + +

Method Summary

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`abort(Transaction tx)`
`void`	`canCommit(long txId, + Collection<byte[]> changeIds)`
`Transaction`	`checkpoint(Transaction originalTx)`
`void`	`commit(long txId, + long writePointer)`
`void`	`doStart()`
`void`	`doStop()`
`TransactionSnapshot`	`getCurrentState()`
`int`	`getExcludedListSize()`
`int`	`getInvalidSize()`
`TransactionSnapshot`	`getSnapshot()`
`TransactionStateStorage`	`getTransactionStateStorage()`
`static long`	`getTxExpirationFromWritePointer(long writePointer, + long timeoutInSeconds)`
`boolean`	`invalidate(long tx)`
`void`	`logStatistics()` + Called from the tx service every 10 seconds. +
`void`	`recoverState()`
`void`	`resetState()` + Resets the state of the transaction manager. +
`Transaction`	`startLong()` + Start a long transaction. +
`Transaction`	`startLong(String clientId)` + Starts a long transaction with a client id. +
`Transaction`	`startShort()` + Start a short transaction with the default timeout. +
`Transaction`	`startShort(int timeoutInSeconds)` + Start a short transaction with a given timeout. +
`Transaction`	`startShort(String clientId)` + Start a short transaction with a client id and default timeout. +
`Transaction`	`startShort(String clientId, + int timeoutInSeconds)` + Start a short transaction with a given timeout. +
`boolean`	`takeSnapshot(OutputStream out)` + Take a snapshot of the transaction state and serialize it into the given output stream. +
`boolean`	`truncateInvalidTx(Set<Long> invalidTxIds)` + Removes the given transaction ids from the invalid list. +
`boolean`	`truncateInvalidTxBefore(long time)` + Removes all transaction ids started before the given time from invalid list. +
`static Map<Long,TransactionManager.InProgressTx>`	`txnBackwardsCompatCheck(int defaultLongTimeout, + long longTimeoutTolerance, + Map<Long,TransactionManager.InProgressTx> inProgress)` + Check if in-progress transactions need to be migrated to have expiration time and type, if so do the migration. +

+ + +
Methods inherited from class com.google.common.util.concurrent.AbstractService
+addListener, isRunning, notifyFailed, notifyStarted, notifyStopped, start, startAndWait, state, stop, stopAndWait, toString

+ + +
Methods inherited from class java.lang.Object
+clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

+ +
- + + +
  Constructor Detail
  + + + +
  - +
    TransactionManager
    +
```
public TransactionManager(org.apache.hadoop.conf.Configuration config)
```
    +
  + + + +
  - +
    TransactionManager
    +
```
@Inject
+public TransactionManager(org.apache.hadoop.conf.Configuration conf,
+                                  @Nonnull
+                                  TransactionStateStorage persistor,
+                                  MetricsCollector txMetricsCollector)
```
    +
  +
+ +
- + + +
  Method Detail
  + + + +
  - +
    doStart
    +
```
public void doStart()
```
    +
    +
    Specified by:
    +
    doStart in class com.google.common.util.concurrent.AbstractService
    +
    +
  + + + +
  - +
    getSnapshot
    +
```
public TransactionSnapshot getSnapshot()
+                                throws IOException
```
    +
    +
    Throws:
    +
    IOException
    +
    +
  + + + +
  - +
    takeSnapshot
    +
```
public boolean takeSnapshot(OutputStream out)
+                     throws IOException
```
    +
    Take a snapshot of the transaction state and serialize it into the given output stream.
    +
    +
    Returns:
    +
    whether a snapshot was taken.
    +
    Throws:
    +
    IOException
    +
    +
  + + + +
  - +
    getCurrentState
    +
```
public TransactionSnapshot getCurrentState()
```
    +
  + + + +
  - +
    recoverState
    +
```
public void recoverState()
```
    +
  + + + +
  - +
    txnBackwardsCompatCheck
    +
```
public static Map<Long,TransactionManager.InProgressTx> txnBackwardsCompatCheck(int defaultLongTimeout,
+                                                                                long longTimeoutTolerance,
+                                                                                Map<Long,TransactionManager.InProgressTx> inProgress)
```
    +
    Check if in-progress transactions need to be migrated to have expiration time and type, if so do the migration. + This is required for backwards compatibility, when long running transactions were represented + with expiration time -1. This can be removed when we stop supporting SnapshotCodec version 1.
    +
  + + + +
  - +
    resetState
    +
```
public void resetState()
```
    +
    Resets the state of the transaction manager.
    +
  + + + +
  - +
    doStop
    +
```
public void doStop()
```
    +
    +
    Specified by:
    +
    doStop in class com.google.common.util.concurrent.AbstractService
    +
    +
  + + + +
  - +
    startShort
    +
```
public Transaction startShort()
```
    +
    Start a short transaction with the default timeout.
    +
  + + + +
  - +
    startShort
    +
```
public Transaction startShort(String clientId)
```
    +
    Start a short transaction with a client id and default timeout.
    +
    +
    Parameters:
    +
    clientId - id of the client requesting a transaction.
    +
    +
  + + + +
  - +
    startShort
    +
```
public Transaction startShort(int timeoutInSeconds)
```
    +
    Start a short transaction with a given timeout.
    +
    +
    Parameters:
    +
    timeoutInSeconds - the time out period in seconds.
    +
    +
  + + + +
  - +
    startShort
    +
```
public Transaction startShort(String clientId,
+                              int timeoutInSeconds)
```
    +
    Start a short transaction with a given timeout.
    +
    +
    Parameters:
    +
    clientId - id of the client requesting a transaction.
    +
    timeoutInSeconds - the time out period in seconds.
    +
    +
  + + + +
  - +
    getTxExpirationFromWritePointer
    +
```
public static long getTxExpirationFromWritePointer(long writePointer,
+                                                   long timeoutInSeconds)
```
    +
  + + + +
  - +
    startLong
    +
```
public Transaction startLong()
```
    +
    Start a long transaction. Long transactions and do not participate in conflict detection. Also, aborting a long + transaction moves it to the invalid list because we assume that its writes cannot be rolled back.
    +
  + + + +
  - +
    startLong
    +
```
public Transaction startLong(String clientId)
```
    +
    Starts a long transaction with a client id.
    +
  + + + +
  - +
    canCommit
    +
```
public void canCommit(long txId,
+                      Collection<byte[]> changeIds)
+               throws TransactionNotInProgressException,
+                      TransactionSizeException,
+                      TransactionConflictException
```
    +
    +
    Throws:
    +
    TransactionNotInProgressException
    +
    TransactionSizeException
    +
    TransactionConflictException
    +
    +
  + + + +
  - +
    commit
    +
```
public void commit(long txId,
+                   long writePointer)
+            throws TransactionNotInProgressException,
+                   TransactionConflictException
```
    +
    +
    Throws:
    +
    TransactionNotInProgressException
    +
    TransactionConflictException
    +
    +
  + + + +
  - +
    abort
    +
```
public void abort(Transaction tx)
```
    +
  + + + +
  - +
    invalidate
    +
```
public boolean invalidate(long tx)
```
    +
  + + + +
  - +
    truncateInvalidTx
    +
```
public boolean truncateInvalidTx(Set<Long> invalidTxIds)
```
    +
    Removes the given transaction ids from the invalid list.
    +
    +
    Parameters:
    +
    invalidTxIds - transaction ids
    +
    Returns:
    +
    true if invalid list got changed, false otherwise
    +
    +
  + + + +
  - +
    truncateInvalidTxBefore
    +
```
public boolean truncateInvalidTxBefore(long time)
+                                throws InvalidTruncateTimeException
```
    +
    Removes all transaction ids started before the given time from invalid list.
    +
    +
    Parameters:
    +
    time - time in milliseconds
    +
    Returns:
    +
    true if invalid list got changed, false otherwise
    +
    Throws:
    +
    InvalidTruncateTimeException - if there are any in-progress transactions started before given time
    +
    +
  + + + +
  - +
    checkpoint
    +
```
public Transaction checkpoint(Transaction originalTx)
+                       throws TransactionNotInProgressException
```
    +
    +
    Throws:
    +
    TransactionNotInProgressException
    +
    +
  + + + +
  - +
    getExcludedListSize
    +
```
public int getExcludedListSize()
```
    +
  + + + +
  - +
    getInvalidSize
    +
```
public int getInvalidSize()
```
    +
    +
    Returns:
    +
    the size of invalid list
    +
    +
  + + + +
  - +
    logStatistics
    +
```
public void logStatistics()
```
    +
    Called from the tx service every 10 seconds. + This hack is needed because current metrics system is not flexible when it comes to adding new metrics.
    +
  + + + +
  - +
    getTransactionStateStorage
    +
```
public TransactionStateStorage getTransactionStateStorage()
```
    +
  +
+

Constructors
Constructor and Description
`TransactionNotInProgressException(String message)`

java.lang.Object
+
- org.apache.tephra.TransactionServiceMain
+

+
+
+
```
public class TransactionServiceMain
+extends Object
```
+
Driver class to start and stop tx in distributed mode.
+

+ +

+ + +
Constructor Summary
+ + + + + + + + + + + +
Constructors
Constructor and Description
TransactionServiceMain()
TransactionServiceMain(org.apache.hadoop.conf.Configuration conf)
+

Constructors
Constructor and Description
`TransactionServiceMain()`
`TransactionServiceMain(org.apache.hadoop.conf.Configuration conf)`

+ +

+ + +

Method Summary

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`destroy()` + Invoked by jsvc for resource cleanup. +
`void`	`doMain(String[] args)` + The main method. +
`void`	`init(String[] args)` + Invoked by jsvc to initialize the program. +
`static void`	`main(String[] args)`
`void`	`start()` + Invoked by jsvc to start the program. +
`void`	`stop()` + Invoked by jsvc to stop the program. +

+ + +
Methods inherited from class java.lang.Object
+clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

+ +
- + + +
  Constructor Detail
  + + + +
  - +
    TransactionServiceMain
    +
```
public TransactionServiceMain()
```
    +
  + + + +
  - +
    TransactionServiceMain
    +
```
public TransactionServiceMain(org.apache.hadoop.conf.Configuration conf)
```
    +
  +
+ +
- + + +
  Method Detail
  + + + +
  - +
    main
    +
```
public static void main(String[] args)
+                 throws Exception
```
    +
    +
    Throws:
    +
    Exception
    +
    +
  + + + +
  - +
    doMain
    +
```
public void doMain(String[] args)
+            throws Exception
```
    +
    The main method. It simply call methods in the same sequence + as if the program is started by jsvc.
    +
    +
    Throws:
    +
    Exception
    +
    +
  + + + +
  - +
    init
    +
```
public void init(String[] args)
```
    +
    Invoked by jsvc to initialize the program.
    +
  + + + +
  - +
    start
    +
```
public void start()
+           throws Exception
```
    +
    Invoked by jsvc to start the program.
    +
    +
    Throws:
    +
    Exception
    +
    +
  + + + +
  - +
    stop
    +
```
public void stop()
```
    +
    Invoked by jsvc to stop the program.
    +
  + + + +
  - +
    destroy
    +
```
public void destroy()
```
    +
    Invoked by jsvc for resource cleanup.
    +
  +
+

Class TransactionManager

Nested Class Summary

Nested classes/interfaces inherited from interface com.google.common.util.concurrent.Service

Constructor Summary

Method Summary

Methods inherited from class com.google.common.util.concurrent.AbstractService

Methods inherited from class java.lang.Object

Constructor Detail

TransactionManager

TransactionManager

Method Detail

doStart

getSnapshot

takeSnapshot

getCurrentState

recoverState

txnBackwardsCompatCheck

resetState

doStop

startShort

startShort

startShort

startShort

getTxExpirationFromWritePointer

startLong

startLong

canCommit

commit

abort

invalidate

truncateInvalidTx

truncateInvalidTxBefore

checkpoint

getExcludedListSize

getInvalidSize

logStatistics

getTransactionStateStorage

Class TransactionNotInProgressException

Constructor Summary

Method Summary

Methods inherited from class java.lang.Throwable

Methods inherited from class java.lang.Object

Constructor Detail

TransactionNotInProgressException

Class TransactionServiceMain

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

TransactionServiceMain

TransactionServiceMain

Method Detail

main

doMain

init

start

stop

destroy