### Description

Allow an Airflow DAG to define a Databricks job with the `api/2.1/jobs/create` (or `api/2.1/jobs/reset`) endpoint then run that same job with the `api/2.1/jobs/run-now` endpoint. This would give similar capabilities as the DatabricksSubmitRun operator, but the `api/2.1/jobs/create` endpoint supports additional parameters that the `api/2.1/jobs/runs/submit` doesn't (e.g. `job_clusters`, `email_notifications`, etc.).

### Use case/motivation

Create and run a Databricks job all in the Airflow DAG. Currently, DatabricksSubmitRun operator uses the `api/2.1/jobs/runs/submit` endpoint which doesn't support all features and creates runs that aren't tied to a job in the Databricks UI. Also, DatabricksRunNow operator requires you to define the job either directly in the Databricks UI or through a separate CI/CD pipeline causing the headache of having to change code in multiple places.

### Related issues

_No response_

### Are you willing to submit a PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

"""True if the current state is a terminal state."""
        return self.state in ("TERMINATING", "TERMINATED", "ERROR", "UNKNOWN")
    @property
    def is_running(self) -> bool:
        """True if the current state is running."""
        return self.state in ("RUNNING", "RESIZING")
    def __eq__(self, other) -> bool:
        return self.state == other.state and self.state_message == other.state_message
    def __repr__(self) -> str:
        return str(self.__dict__)
    def to_json(self) -> str:
        return json.dumps(self.__dict__)
    @classmethod
    def from_json(cls, data: str) -> ClusterState:
        return ClusterState(**json.loads(data))
class DatabricksHook(BaseDatabricksHook):
    """
    Interact with Databricks.
    :param databricks_conn_id: Reference to the :ref:`Databricks connection <howto/connection:databricks>`.
    :param timeout_seconds: The amount of time in seconds the requests library
        will wait before timing-out.
    :param retry_limit: The number of times to retry the connection in case of
        service outages.
    :param retry_delay: The number of seconds to wait between retries (it
        might be a floating point number).
    :param retry_args: An optional dictionary with arguments passed to ``tenacity.Retrying`` class.
    """
    hook_name = "Databricks"
    def __init__(