org.apache.flink.yarn

类 YarnResourceManager

java.lang.Object

org.apache.flink.runtime.rpc.RpcEndpoint

org.apache.flink.runtime.rpc.FencedRpcEndpoint<org.apache.flink.runtime.resourcemanager.ResourceManagerId>

org.apache.flink.runtime.resourcemanager.ResourceManager<WorkerType>

org.apache.flink.runtime.resourcemanager.ActiveResourceManager<YarnWorkerNode>

org.apache.flink.yarn.YarnResourceManager

所有已实现的接口:

AutoCloseable, org.apache.flink.runtime.leaderelection.LeaderContender, org.apache.flink.runtime.resourcemanager.ResourceManagerGateway, org.apache.flink.runtime.rpc.FencedRpcGateway<org.apache.flink.runtime.resourcemanager.ResourceManagerId>, org.apache.flink.runtime.rpc.RpcGateway, org.apache.flink.util.AutoCloseableAsync, org.apache.hadoop.yarn.client.api.async.AMRMClientAsync.CallbackHandler, org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler

public class YarnResourceManager
extends org.apache.flink.runtime.resourcemanager.ActiveResourceManager<YarnWorkerNode>
implements org.apache.hadoop.yarn.client.api.async.AMRMClientAsync.CallbackHandler, org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler

The yarn implementation of the resource manager. Used when the system is started via the resource framework YARN.

嵌套类概要

从类继承的嵌套类/接口 org.apache.flink.runtime.rpc.RpcEndpoint

org.apache.flink.runtime.rpc.RpcEndpoint.MainThreadExecutor

字段概要

从类继承的字段 org.apache.flink.runtime.resourcemanager.ActiveResourceManager

defaultMemoryMB, env, flinkClientConfig, flinkConfig, numSlotsPerTaskManager, resourceProfilesPerWorker, taskExecutorProcessSpec

从类继承的字段 org.apache.flink.runtime.resourcemanager.ResourceManager

RESOURCE_MANAGER_NAME

从类继承的字段 org.apache.flink.runtime.rpc.RpcEndpoint

log, rpcServer

构造器概要

构造器

构造器和说明

YarnResourceManager(org.apache.flink.runtime.rpc.RpcService rpcService, String resourceManagerEndpointId, org.apache.flink.runtime.clusterframework.types.ResourceID resourceId, org.apache.flink.configuration.Configuration flinkConfig, Map<String,String> env, org.apache.flink.runtime.highavailability.HighAvailabilityServices highAvailabilityServices, org.apache.flink.runtime.heartbeat.HeartbeatServices heartbeatServices, org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager slotManager, org.apache.flink.runtime.resourcemanager.JobLeaderIdService jobLeaderIdService, org.apache.flink.runtime.entrypoint.ClusterInformation clusterInformation, org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler, String webInterfaceUrl, org.apache.flink.runtime.metrics.groups.ResourceManagerMetricGroup resourceManagerMetricGroup)

方法概要

限定符和类型	方法和说明
protected org.apache.hadoop.yarn.client.api.async.NMClientAsync	createAndStartNodeManagerClient(org.apache.hadoop.yarn.conf.YarnConfiguration yarnConfiguration)
protected org.apache.hadoop.yarn.client.api.async.AMRMClientAsync<org.apache.hadoop.yarn.client.api.AMRMClient.ContainerRequest>	createAndStartResourceManagerClient(org.apache.hadoop.yarn.conf.YarnConfiguration yarnConfiguration, int yarnHeartbeatIntervalMillis, String webInterfaceUrl)
protected double	getCpuCores(org.apache.flink.configuration.Configuration configuration)
float	getProgress()
protected void	initialize()
protected void	internalDeregisterApplication(org.apache.flink.runtime.clusterframework.ApplicationStatus finalStatus, String diagnostics)
protected org.apache.flink.configuration.Configuration	loadClientConfiguration()
void	onContainersAllocated(List<org.apache.hadoop.yarn.api.records.Container> containers)
void	onContainersCompleted(List<org.apache.hadoop.yarn.api.records.ContainerStatus> statuses)
void	onContainerStarted(org.apache.hadoop.yarn.api.records.ContainerId containerId, Map<String,ByteBuffer> map)
void	onContainerStatusReceived(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.ContainerStatus containerStatus)
void	onContainerStopped(org.apache.hadoop.yarn.api.records.ContainerId containerId)
void	onError(Throwable error)
void	onGetContainerStatusError(org.apache.hadoop.yarn.api.records.ContainerId containerId, Throwable throwable)
void	onNodesUpdated(List<org.apache.hadoop.yarn.api.records.NodeReport> list)
void	onShutdownRequest()
void	onStartContainerError(org.apache.hadoop.yarn.api.records.ContainerId containerId, Throwable t)
CompletableFuture<Void>	onStop()
void	onStopContainerError(org.apache.hadoop.yarn.api.records.ContainerId containerId, Throwable throwable)
Collection<org.apache.flink.runtime.clusterframework.types.ResourceProfile>	startNewWorker(org.apache.flink.runtime.clusterframework.types.ResourceProfile resourceProfile)
boolean	stopWorker(YarnWorkerNode workerNode)
protected YarnWorkerNode	workerStarted(org.apache.flink.runtime.clusterframework.types.ResourceID resourceID)

从类继承的方法 org.apache.flink.runtime.resourcemanager.ActiveResourceManager

getStopTerminationFutureOrCompletedExceptionally

从类继承的方法 org.apache.flink.runtime.resourcemanager.ResourceManager

cancelSlotRequest, clearStateAsync, closeJobManagerConnection, closeTaskManagerConnection, deregisterApplication, disconnectJobManager, disconnectTaskManager, getNumberOfRegisteredTaskManagers, getNumberRequiredTaskManagerSlots, grantLeadership, handleError, heartbeatFromJobManager, heartbeatFromTaskManager, jobLeaderLostLeadership, notifySlotAvailable, onFatalError, onStart, prepareLeadershipAsync, registerJobManager, registerTaskExecutor, releaseResource, removeJob, requestResourceOverview, requestSlot, requestTaskManagerFileUpload, requestTaskManagerInfo, requestTaskManagerInfo, requestTaskManagerMetricQueryServiceAddresses, revokeLeadership, sendSlotReport, setFailUnfulfillableRequest, startServicesOnLeadership

从类继承的方法 org.apache.flink.runtime.rpc.FencedRpcEndpoint

callAsyncWithoutFencing, getFencingToken, getMainThreadExecutor, getUnfencedMainThreadExecutor, runAsyncWithoutFencing, setFencingToken

从类继承的方法 org.apache.flink.runtime.rpc.RpcEndpoint

callAsync, closeAsync, getAddress, getEndpointId, getHostname, getRpcService, getSelfGateway, getTerminationFuture, internalCallOnStart, internalCallOnStop, isRunning, runAsync, scheduleRunAsync, scheduleRunAsync, start, stop, validateRunsInMainThread

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.runtime.rpc.FencedRpcGateway

getFencingToken

从接口继承的方法 org.apache.flink.runtime.rpc.RpcGateway

getAddress, getHostname

从接口继承的方法 org.apache.flink.runtime.leaderelection.LeaderContender

getDescription

从接口继承的方法 org.apache.flink.util.AutoCloseableAsync

close

构造器详细资料

YarnResourceManager

public YarnResourceManager(org.apache.flink.runtime.rpc.RpcService rpcService,
                           String resourceManagerEndpointId,
                           org.apache.flink.runtime.clusterframework.types.ResourceID resourceId,
                           org.apache.flink.configuration.Configuration flinkConfig,
                           Map<String,String> env,
                           org.apache.flink.runtime.highavailability.HighAvailabilityServices highAvailabilityServices,
                           org.apache.flink.runtime.heartbeat.HeartbeatServices heartbeatServices,
                           org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager slotManager,
                           org.apache.flink.runtime.resourcemanager.JobLeaderIdService jobLeaderIdService,
                           org.apache.flink.runtime.entrypoint.ClusterInformation clusterInformation,
                           org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler,
                           @Nullable
                           String webInterfaceUrl,
                           org.apache.flink.runtime.metrics.groups.ResourceManagerMetricGroup resourceManagerMetricGroup)

方法详细资料

createAndStartResourceManagerClient

protected org.apache.hadoop.yarn.client.api.async.AMRMClientAsync<org.apache.hadoop.yarn.client.api.AMRMClient.ContainerRequest> createAndStartResourceManagerClient(org.apache.hadoop.yarn.conf.YarnConfiguration yarnConfiguration,
                                                                                                                                                                     int yarnHeartbeatIntervalMillis,
                                                                                                                                                                     @Nullable
                                                                                                                                                                     String webInterfaceUrl)
                                                                                                                                                              throws Exception

抛出:

Exception

createAndStartNodeManagerClient

protected org.apache.hadoop.yarn.client.api.async.NMClientAsync createAndStartNodeManagerClient(org.apache.hadoop.yarn.conf.YarnConfiguration yarnConfiguration)

loadClientConfiguration

protected org.apache.flink.configuration.Configuration loadClientConfiguration()

指定者:

loadClientConfiguration 在类中 org.apache.flink.runtime.resourcemanager.ActiveResourceManager<YarnWorkerNode>

initialize

protected void initialize()
                   throws org.apache.flink.runtime.resourcemanager.exceptions.ResourceManagerException

指定者:

initialize 在类中 org.apache.flink.runtime.resourcemanager.ResourceManager<YarnWorkerNode>

抛出:

org.apache.flink.runtime.resourcemanager.exceptions.ResourceManagerException

onStop

public CompletableFuture<Void> onStop()

覆盖:

onStop 在类中 org.apache.flink.runtime.resourcemanager.ResourceManager<YarnWorkerNode>

internalDeregisterApplication

protected void internalDeregisterApplication(org.apache.flink.runtime.clusterframework.ApplicationStatus finalStatus,
                                             @Nullable
                                             String diagnostics)

指定者:

internalDeregisterApplication 在类中 org.apache.flink.runtime.resourcemanager.ResourceManager<YarnWorkerNode>

startNewWorker

public Collection<org.apache.flink.runtime.clusterframework.types.ResourceProfile> startNewWorker(org.apache.flink.runtime.clusterframework.types.ResourceProfile resourceProfile)

指定者:

startNewWorker 在类中 org.apache.flink.runtime.resourcemanager.ResourceManager<YarnWorkerNode>

stopWorker

public boolean stopWorker(YarnWorkerNode workerNode)

指定者:

stopWorker 在类中 org.apache.flink.runtime.resourcemanager.ResourceManager<YarnWorkerNode>

workerStarted

protected YarnWorkerNode workerStarted(org.apache.flink.runtime.clusterframework.types.ResourceID resourceID)

指定者:

workerStarted 在类中 org.apache.flink.runtime.resourcemanager.ResourceManager<YarnWorkerNode>

getProgress

public float getProgress()

指定者:

getProgress 在接口中 org.apache.hadoop.yarn.client.api.async.AMRMClientAsync.CallbackHandler

onContainersCompleted

public void onContainersCompleted(List<org.apache.hadoop.yarn.api.records.ContainerStatus> statuses)

指定者:

onContainersCompleted 在接口中 org.apache.hadoop.yarn.client.api.async.AMRMClientAsync.CallbackHandler

onContainersAllocated

public void onContainersAllocated(List<org.apache.hadoop.yarn.api.records.Container> containers)

指定者:

onContainersAllocated 在接口中 org.apache.hadoop.yarn.client.api.async.AMRMClientAsync.CallbackHandler

onShutdownRequest

public void onShutdownRequest()

指定者:

onShutdownRequest 在接口中 org.apache.hadoop.yarn.client.api.async.AMRMClientAsync.CallbackHandler

onNodesUpdated

public void onNodesUpdated(List<org.apache.hadoop.yarn.api.records.NodeReport> list)

指定者:

onNodesUpdated 在接口中 org.apache.hadoop.yarn.client.api.async.AMRMClientAsync.CallbackHandler

onError

public void onError(Throwable error)

指定者:

onError 在接口中 org.apache.hadoop.yarn.client.api.async.AMRMClientAsync.CallbackHandler

onContainerStarted

public void onContainerStarted(org.apache.hadoop.yarn.api.records.ContainerId containerId,
                               Map<String,ByteBuffer> map)

指定者:

onContainerStarted 在接口中 org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler

onContainerStatusReceived

public void onContainerStatusReceived(org.apache.hadoop.yarn.api.records.ContainerId containerId,
                                      org.apache.hadoop.yarn.api.records.ContainerStatus containerStatus)

指定者:

onContainerStatusReceived 在接口中 org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler

onContainerStopped

public void onContainerStopped(org.apache.hadoop.yarn.api.records.ContainerId containerId)

指定者:

onContainerStopped 在接口中 org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler

onStartContainerError

public void onStartContainerError(org.apache.hadoop.yarn.api.records.ContainerId containerId,
                                  Throwable t)

指定者:

onStartContainerError 在接口中 org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler

onGetContainerStatusError

public void onGetContainerStatusError(org.apache.hadoop.yarn.api.records.ContainerId containerId,
                                      Throwable throwable)

指定者:

onGetContainerStatusError 在接口中 org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler

onStopContainerError

public void onStopContainerError(org.apache.hadoop.yarn.api.records.ContainerId containerId,
                                 Throwable throwable)

指定者:

onStopContainerError 在接口中 org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler

getCpuCores

protected double getCpuCores(org.apache.flink.configuration.Configuration configuration)

指定者:

getCpuCores 在类中 org.apache.flink.runtime.resourcemanager.ActiveResourceManager<YarnWorkerNode>