For conventional thermally conductive polymer composites (TCPCs), very high loading of conventional thermally conductive fillers (> 30 vol%) is usually needed to obtain appropriate thermal conductivity, which reduces product quality through diminished mechanical properties, increased weight, reduced flexibility, increased fragility, and increased the difficulty and cost of production. Thus, a method of TCPC fabrication with high thermal conductivity at low filler loading is needed. In the current study, adsorption-desorption of polymer chains on the graphene (GE) surface was used to trap GE at the interface of an immiscible poly(ε-caprolactone) (PCL)/poly(lactic acid) (PLA) blend. The resulting composite exhibits remarkably high thermal conductivity due to the graphene sheets being controlled at the interface of the polymer blend. At 0.53 vol%, when a good co-continuous structure is formed, the thermal conductivity of the graphene composite is nearly 4 times higher than that of the pure PCL/PLA blend with co-continuous structure. Moreover, the blend achieves an extremely low thermal percolation threshold (0.11 vol%), the lowest to date, because most of the graphene sheets are selectively located at the interface of the blend with co-continuous structure.