We use 3D contact dynamics simulations to analyze the rheological properties of granular materials composed of rigid aggregates. The aggregates are made from four overlapping spheres and described by a nonconvexity parameter depending on the relative positions of the spheres. The macroscopic and microstructural properties of several sheared packings are analyzed as a function of the degree of nonconvexity of the aggregates. We find that the internal angle of friction increases with nonconvexity. In contrast, the packing fraction increases first to a maximum value but declines as nonconvexity further increases. At high level of nonconvexity, the packings are looser but show a higher shear strength. At the microscopic scale, the fabric and force anisotropy, as well as friction mobilization are enhanced by multiple contacts between aggregates and interlocking, revealing thus the mechanical and geometrical origins of shear strength.