A melt compounding method to achieve the exfoliation of layered zirconium phosphate (ZrP) nanoplatelets was presented in this work. ZrP nanoplatelets were first intercalated by diglycolamine (DGA) to increase the interlayer distance and functionality. In the next melt compounding process, the cyclic anhydrides of maleic anhydride grafted polyolefin elastomers (POE-g-MA) reacted with the hydroxyls in DGA-ZrP. Subsequently, the long polyolefin chains of POE-g-MA intercalated into the interlayer of ZrP smoothly and exfoliated the nanoplatelets successfully. ZrP nanoplatelets with various sizes were fully exfoliated in POE-g-MA matrices. The modulus of elasticity of POE-g-MA was improved by the incorporation of ZrP nanoplatelets and further increased if the nanoplatelets are exfoliated. The ductility of ZrP/POE-g-MA compounds also became better, if ZrP nanoplatelets were first intercalated by DGA and then exfoliated, especially for the smaller nanoplatelets. Our work provides a general method to prepare polymer nanocomposites containing exfoliated ZrP nanoplatelets in a large scale.