### Description

Allow an Airflow DAG to define a Databricks job with the `api/2.1/jobs/create` (or `api/2.1/jobs/reset`) endpoint then run that same job with the `api/2.1/jobs/run-now` endpoint. This would give similar capabilities as the DatabricksSubmitRun operator, but the `api/2.1/jobs/create` endpoint supports additional parameters that the `api/2.1/jobs/runs/submit` doesn't (e.g. `job_clusters`, `email_notifications`, etc.).

### Use case/motivation

Create and run a Databricks job all in the Airflow DAG. Currently, DatabricksSubmitRun operator uses the `api/2.1/jobs/runs/submit` endpoint which doesn't support all features and creates runs that aren't tied to a job in the Databricks UI. Also, DatabricksRunNow operator requires you to define the job either directly in the Databricks UI or through a separate CI/CD pipeline causing the headache of having to change code in multiple places.

### Related issues

_No response_

### Are you willing to submit a PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

validate_trigger_event(event)
    run_state = RunState.from_json(event["run_state"])
    run_page_url = event["run_page_url"]
    log.info("View run status, Spark UI, and logs at %s", run_page_url)
    if run_state.is_successful:
        log.info("Job run completed successfully.")
        return
    else:
        error_message = f"Job run failed with terminal state: {run_state}"
        raise AirflowException(error_message)
class DatabricksJobRunLink(BaseOperatorLink):
    """Constructs a link to monitor a Databricks Job Run."""
    name = "See Databricks Job Run"
    def get_link(
        self,
        operator: BaseOperator,
        *,
        ti_key: TaskInstanceKey,
    ) -> str:
        return XCom.get_value(key=XCOM_RUN_PAGE_URL_KEY, ti_key=ti_key)
class DatabricksSubmitRunOperator(BaseOperator):