Bug: [REFACTOR]: handle network delays with expiry of access token



It may so happen that there might be a slight delay in the network to receive the access token from connector. So when we store the access token with expiry already few seconds would have elapsed. So the key might expire few seconds before than the actual expiry at hyperswitch.

**Note** - This affects only to those connectors which provide relative expiry i.e expires in 300s. 

In order to account for this, when storing the redis expiry key, it can be set by subtracting few seconds from the actual expiry in order to account for the network delays.

**Note** - This can be done in the core level for all access token in order to be safe. There might also be some network delays when using this access token to create the payment. The token might have not been expired when the request is sent, but once it reaches the processor, it might have been expired.
diff --git a/crates/router/src/consts.rs b/crates/router/src/consts.rs
index 0c9e9443ac9..6d9da8f7bb5 100644
--- a/crates/router/src/consts.rs
+++ b/crates/router/src/consts.rs
@@ -118,6 +118,8 @@ pub const POLL_ID_TTL: i64 = 900;
 pub const DEFAULT_POLL_DELAY_IN_SECS: i8 = 2;
 pub const DEFAULT_POLL_FREQUENCY: i8 = 5;
 
+// Number of seconds to subtract from access token expiry
+pub(crate) const REDUCE_ACCESS_TOKEN_EXPIRY_TIME: u8 = 15;
 pub const CONNECTOR_CREDS_TOKEN_TTL: i64 = 900;
 
 //max_amount allowed is 999999999 in minor units
diff --git a/crates/router/src/core/payments/access_token.rs b/crates/router/src/core/payments/access_token.rs
index 2d4ed2b6e1b..0c5aa13eb2f 100644
--- a/crates/router/src/core/payments/access_token.rs
+++ b/crates/router/src/core/payments/access_token.rs
@@ -91,9 +91,26 @@ pub async fn add_access_token<
                     connector.connector_name,
                     access_token.expires
                 );
+                metrics::ACCESS_TOKEN_CACHE_HIT.add(
+                    &metrics::CONTEXT,
+                    1,
+                    &[metrics::request::add_attributes(
+                        "connector",
+                        connector.connector_name.to_string(),
+                    )],
+                );
                 Ok(Some(access_token))
             }
             None => {
+                metrics::ACCESS_TOKEN_CACHE_MISS.add(
+                    &metrics::CONTEXT,
+                    1,
+                    &[metrics::request::add_attributes(
+                        "connector",
+                        connector.connector_name.to_string(),
+                    )],
+                );
+
                 let cloned_router_data = router_data.clone();
                 let refresh_token_request_data = types::AccessTokenRequestData::try_from(
                     router_data.connector_auth_type.clone(),
@@ -123,15 +140,31 @@ pub async fn add_access_token<
                     &refresh_token_router_data,
                 )
                 .await?
-                .async_map(|access_token| async {
-                    // Store the access token in redis with expiry
-                    // The expiry should be adjusted for network delays from the connector
+                .async_map(|access_token| async move {
                     let store = &*state.store;
+
+                    // The expiry should be adjusted for network delays from the connector
+                    // The access token might not have been expired when request is sent
+                    // But once it reaches the connector, it might expire because of the network delay
+                    // Subtract few seconds from the expiry in order to account for these network delays
+                    // This will reduce the expiry time by `REDUCE_ACCESS_TOKEN_EXPIRY_TIME` seconds
+                    let modified_access_token_with_expiry = types::AccessToken {
+                        expires: access_token
+                            .expires
+                            .saturating_sub(consts::REDUCE_ACCESS_TOKEN_EXPIRY_TIME.into()),
+                        ..access_token
+                    };
+
+                    logger::debug!(
+                        access_token_expiry_after_modification =
+                            modified_access_token_with_expiry.expires
+                    );
+
                     if let Err(access_token_set_error) = store
                         .set_access_token(
                             merchant_id,
                             &merchant_connector_id_or_connector_name,
-                            access_token.clone(),
+                            modified_access_token_with_expiry.clone(),
                         )
                         .await
                         .change_context(errors::ApiErrorResponse::InternalServerError)
@@ -142,7 +175,7 @@ pub async fn add_access_token<
                         // The next request will create new access token, if required
                         logger::error!(access_token_set_error=?access_token_set_error);
                     }
-                    Some(access_token)
+                    Some(modified_access_token_with_expiry)
                 })
                 .await
             }
diff --git a/crates/router/src/routes/metrics.rs b/crates/router/src/routes/metrics.rs
index 780dd51bbde..1123be1a874 100644
--- a/crates/router/src/routes/metrics.rs
+++ b/crates/router/src/routes/metrics.rs
@@ -52,7 +52,6 @@ counter_metric!(MCA_CREATE, GLOBAL_METER);
 
 // Flow Specific Metrics
 
-counter_metric!(ACCESS_TOKEN_CREATION, GLOBAL_METER);
 histogram_metric!(CONNECTOR_REQUEST_TIME, GLOBAL_METER);
 counter_metric!(SESSION_TOKEN_CREATED, GLOBAL_METER);
 
@@ -123,5 +122,16 @@ counter_metric!(TASKS_ADDED_COUNT, GLOBAL_METER); // Tasks added to process trac
 counter_metric!(TASK_ADDITION_FAILURES_COUNT, GLOBAL_METER); // Failures in task addition to process tracker
 counter_metric!(TASKS_RESET_COUNT, GLOBAL_METER); // Tasks reset in process tracker for requeue flow
 
+// Access token metrics
+//
+// A counter to indicate the number of new access tokens created
+counter_metric!(ACCESS_TOKEN_CREATION, GLOBAL_METER);
+
+// A counter to indicate the access token cache hits
+counter_metric!(ACCESS_TOKEN_CACHE_HIT, GLOBAL_METER);
+
+// A counter to indicate the access token cache miss
+counter_metric!(ACCESS_TOKEN_CACHE_MISS, GLOBAL_METER);
+
 pub mod request;
 pub mod utils;

## Type of Change
<!-- Put an `x` in the boxes that apply -->

- [x] Refactoring

## Description
<!-- Describe your changes in detail -->
There might be some delay in network due to which the access token which was not expired locally, might have expired when it reaches the connector. This PR reduces the expiry of access token in order to account for such delays

<!--
Provide links to the files with corresponding changes.

Following are the paths where you can find config files:
1. `config`
2. `crates/router/src/configs`
3. `loadtest/config`
-->


## Motivation and Context
<!--
Why is this change required? What problem does it solve?
If it fixes an open issue, please link to the issue here.

If you don't have an issue, we'd recommend starting with one first so the PR
can focus on the implementation (unless it is an obvious bug or documentation fix
that will have little conversation).
-->


## How did you test it?
<!--
Did you write an integration/unit/API test to verify the code changes?
Or did you test this change manually (provide relevant screenshots)?
-->
- Create a payment with any connector which supports access token and then check the ttl in redis.
<img width="2052" alt="Screenshot 2024-05-10 at 12 24 52 AM" src="https://github.com/juspay/hyperswitch/assets/48803246/1ef1d233-d0c0-4f6d-a511-b80e61805513">

- The below log can also be checked
<img width="1284" alt="Screenshot 2024-05-10 at 12 25 58 AM" src="https://github.com/juspay/hyperswitch/assets/48803246/b69f742f-9f22-4960-a446-12e77ac03769">

## IMPACT
- airwallex
- globalpay
- paypal
- payu
- trustpay - bank redirect
- iatapay
- volt

## Checklist
<!-- Put an `x` in the boxes that apply -->

- [x] I formatted the code `cargo +nightly fmt --all`
- [x] I addressed lints thrown by `cargo clippy`
- [x] I reviewed the submitted code
