Monday, January 27, 2014

Main Hadoop 2.0 daemons !

  • NameNode : one per Namespace, stores & handles HDFS metadata
  • Secondary NameNode : (for now) still in use if no HA
  • Checkpoint node : (later) multiple checkpoint node is possible, performs periodic checkpoints
  • Backup node / Standby node : allows high availability, keep updated copy of namespace in its memory (if using no checkpoint allowed)
  • DataNode : stores HDFS data

  • ResourceManager : a global pure scheduler
  • ApplicationMaster : one per application, negociates ressource with RM, monitors and asks task execution to NodeManager 
  • NodeManager : one per slave server, a task application container launcher and reporting agent
  • Application Container : a separate processing unit, it can be a Map, a Reduce or a Storm Bolt, etc

No comments: