This study employs a finite element method based modeling of cementitious composite microstructure to study the effect of presence of inclusions on the stress distribution and the constitutive response of the composite. A randomized periodic microstructure combined with periodic boundary conditions forms the base of the finite element models. Inclusion properties of quartz and light weight aggregates of size 600μm obtained from literature were made use of to study the effect of their material (including inclusion stiffness, stiffness of interfacial transition zone and matrix stiffening) and geometric properties (volume fraction of inclusion, particle size distribution of inclusion and thickness of the interfacial transition zone) on the composite. Traction-separation relationship was used to incorporate the effect of debonding at the interface of the matrix and the inclusion to study the effect on stress distribution in the microstructure. The stress distributions observed upon conducting a finite element analysis are caused due to the stiffness mismatch in both the quartz and the light weight aggregates as expected. The constitutive response of the composite microstructure is found to be in good conformance with semi-analytical models as well as experimental values. The effect of debonding throws up certain important observations on the stress distributions in the microstructure based on the stress concentrations and relaxations caused by the stiffness of the individual components of the microstructure. The study presented discusses the different micromechanical models employed, their applicability and suitability to correctly predict the composite constitutive response.