novatechflow | Alexander Alten

This guide shows how to mount an HDFS filesystem using FUSE and then export part of it over NFS so that remote systems can access HDFS like a local filesystem. The approach is based on classic Hadoop and Linux tools, and includes notes on security, user mapping, and kernel limitations. In some environments, it can be useful to make an HDFS filesystem available across networks as an exported share. This walkthrough describes a working scenario using Linux and Hadoop with tools that are typically included in older Hadoop distributions. The setup uses hadoop-fuse-dfs and libhdfs to mount HDFS locally, and then exports that mount over NFS. Replace namenode.local and <PORT> with values appropriate for your cluster. 1. Install FUSE and libhdfs yum install hadoop-0.20-fuse.x86_64 hadoop-0.20-libhdfs.x86_64 2. Create a mountpoint mkdir /hdfs-mount 3. Test mounting HDFS via FUSE hadoop-fuse-dfs dfs://namenode.local:<PORT> /hdfs-mount -d If the mount succee...

This article explains how to debug runtime issues in classical Hadoop MapReduce clusters using JobTracker stack traces, jps, vmstat, and thread-level CPU analysis. Updated notes show how these same debugging principles apply in modern YARN-based clusters, including ResourceManager, NodeManager and NameNode troubleshooting, improved commands, JMX endpoints, and best practices for memory, networking, and virtualization. In early Hadoop deployments (MRv1), one of the most effective ways to diagnose cluster issues was to inspect JobTracker stack traces and JVM thread states. While modern Hadoop clusters use YARN, the same root causes—network latency, RPC timeouts, NameNode memory pressure, GC stalls, and overloaded system services—still apply today. 1. Legacy Method: Inspect JobTracker Stack Traces In MRv1 clusters, you could view all active JobTracker JVM threads via: http://jobtracker:50030/stacks Example thread dump: Process Thread Dump: 43 active threads Thread 3203101 ...