The general rule for the sentence that begin with by the time is:
'By the time + simple present, future perfect tense'.
So, the use of 'will have closed' is appropriate.
The correct sentence is- 'By the time we reach the library, the book issuing section will have closed.'
Hence, option (c) is correct.
NOTE/-
The simple present is a verb tense used to talk about conditions or actions happening right now or habitual actions and occurrences.
The future perfect is a verb tense used for actions that will be completed before some other point in the future.
For example- The parade will have ended by the time Chester gets out of bed. At eight o'clock I will have left.