Any process that modulates the frequency, rate or extent of hemocyte proliferation. Hemocytes are blood cells associated with a hemocoel (the cavity containing most of the major organs of the arthropod body) which are involved in defense and clotting