data synchronization

User data synchronization

Features

User data synchronization is mainly to synchronize users and organizations between different systems through the SCIM protocol，Use server/Client mode，Server offers a user that meets the SCIM standard protocol，Group and other interfaces，The client side pulls the interface provided by the server through the timing task to obtain the data

Classic scenes:

• ADSynchronization with Arkid
• HRSynchronization with Arkid
• HRSynchronization with AD

SCIMAgreement reference

• RFC7643 - SCIM: Core Schema
• RFC7644 - SCIM: Protocol
• RFC7642 - SCIM: Definitions, Overview, Concepts, and Requirements

Implementation

first，The implementation of the Server side SCIM protocol is implemented in the code Scim_server Module。
in, Three important categories are:

• scim_server.views.view_template.ViewTemplate
  • Subclass Scim_server.views.users_view.UsersViewTemplateProcessing user -related addition, deletion, change check
  • Subclass Scim_server.views.groups_view.GroupsViewTemplateTreatment of related additions, deletion, change inspection
• scim_server.service.provider_adapter_template.ProviderAdapterTemplate
• scim_server.service.provider_base.ProviderBase

SCIM ServerThe approximate process of processing the SCIM request is，ViewTemplateAccept request，Convert the request parameter to an object to pass it toProviderAdapterTemplate，
ProviderAdapterTemplateVerification request parameter legality，And further assemble the request object，Final callProviderBaseThe method of processing the request object。

ScimSyncArkIDExtensionPlug -in base class inheritanceProviderBase，Created when plug -in loadUsersViewandGroupsViewSeparate inheritanceUsersViewTemplateandGroupsViewTemplate，
And register the corresponding users_url and groups_url，At this point, you only needProviderBaseInheritedquery_users, query_groupsSCIM can be implemented by other methods Server。
Create SCIM Call during the server configurationapi.views.Scim_sync.create_Scim_syncInterface processing function，Back to USERS at the same time_url and groups_URL to pick data for the client side

ClientPass through django_celery_Beat creates timing tasks，First callapi.views.Scim_sync.create_Scim_syncThe interface processing function creates the configuration of the client mode，The configuration parameters need to specify sciM Server，Used to from scim Use provided by the server_url and groups_URL pull data，
Determine in the processing function to determine if it is the configuration of the CLIENT mode，Create a regular task，Pass the configuration of the client mode to Celery asynchronous task：arkid.core.tasks.sync, This task will eventually call [Sync] in the plug -in base class (#arkid.core.extension.Scim_sync.ScimSyncExtension.Sync) method，
syncMethods first will be adjusted get_groups_users method to get Users and Groups, Then call SYNC_groups Sync_users to implement synchronous logic，The specific plug -in needs to cover this two methods to achieve the synchronization logic of the client side

Abstract method

ServerModel abstraction method

• create_user
• create_group
• delete_user
• delete_group
• replace_user
• replace_group
• retrieve_user
• retrieve_group
• update_user
• update_group
• query_users
• query_groups

ClientModel abstraction method

• sync_groups
• sync_users

Foundation definition

arkid.core.extension.scim_sync.ScimSyncExtension (Extension, ProviderBase)

Source code in arkid/core/extension/scim_sync.py

class ScimSyncExtension(Extension, ProviderBase):
    TYPE = "scim_sync"

    composite_schema_map = {}
    created_composite_schema_list = []
    composite_key = 'type'
    composite_model = TenantExtensionConfig

    @property
    def type(self):
        return ScimSyncExtension.TYPE

    def load(self):
        class UsersView(UsersViewTemplate):
            @property
            def provider(this):
                return self

            @method_decorator(jwt_token_required)
            def dispatch(self, request, *args, **kwargs):
                return super().dispatch(request, *args, **kwargs)

        class GroupsView(GroupsViewTemplate):
            @property
            def provider(this):
                return self

            @method_decorator(jwt_token_required)
            def dispatch(self, request, *args, **kwargs):
                return super().dispatch(request, *args, **kwargs)

        scim_server_urls = [
            re_path(
                rf'^scim/{self.pname}/(?P<config_id>[\w-]+)/Users(?:/(?P<uuid>[^/]+))?$',
                UsersView.as_view(),
                name=f'{self.pname}_scim_users',
            ),
            # re_path(r'^Groups/.search$', views.GroupSearchView.as_view(), name='groups-search'),
            re_path(
                rf'^scim/{self.pname}/(?P<config_id>[\w-]+)/Groups(?:/(?P<uuid>[^/]+))?$',
                GroupsView.as_view(),
                name=f'{self.pname}_scim_groups',
            ),
        ]
        self.register_routers(scim_server_urls, True)
        super().load()

    def register_scim_sync_schema(self, sync_type, client_schema, server_schema):
        schema = create_extension_schema_by_package(
            self.package,
            fields=[
                (
                    "__root__",
                    Union[(client_schema, server_schema)],
                    Field(discriminator="mode"),
                )
            ],
            base_schema=RootSchema,
        )
        self.register_config_schema(schema, self.package + '_' + sync_type)
        self.register_composite_config_schema(schema, sync_type, exclude=['extension'])

    def sync(self, config, sync_log):
        """
        Args:
            config (arkid.extension.models.TenantExtensionConfig): Client模式创建的配置
        """
        logger.info(
            f"============= Sync Start With Config: {config}/{config.config} ================"
        )
        groups, users = self.get_groups_users(config)
        if not groups or not users:
            return
        self.sync_groups(groups, config, sync_log)
        self.sync_users(users, config, sync_log)

    def get_data(self, url, token):
        logger.info(f"Getting data from {url}")
        headers = {"Authorization": f"jwt {token}"}
        r = requests.get(url, headers=headers)
        if r.status_code == 200:
            return r.json()
        return {}

    def get_groups_users(self, config):
        """
        Args:
            config (arkid.extension.models.TenantExtensionConfig): Client模式创建的配置
        """
        sync_server_id = config.config.get("sync_server", {}).get("id")
        server_config = TenantExtensionConfig.active_objects.filter(
            id=sync_server_id
        ).first()
        if not server_config:
            logger.error(f"No scim sync server config found: {sync_server_id}")
            return None, None
        group_url = server_config.config["group_url"]
        user_url = server_config.config["user_url"]
        token = server_config.config["token"]
        groups = self.get_data(group_url, token).get("Resources")
        users = self.get_data(user_url, token).get("Resources")
        return groups, users

    @abstractmethod
    def sync_groups(self, groups, config, sync_log):
        """
        抽象方法
        Args:
            groups (List): SCIM Server返回的组织列表
            config (arkid.extension.models.TenantExtensionConfig): Client模式创建的配置
        """
        pass

    @abstractmethod
    def sync_users(self, users, config, sync_log):
        """
        抽象方法
        Args:
            users (List): SCIM Server返回的用户列表
            config (arkid.extension.models.TenantExtensionConfig): Client模式创建的配置
        """
        pass

    def get_current_config(self, event):
        config_id = event.request.POST.get('config_id')
        return self.get_config_by_id(config_id)

    def create_tenant_config(self, tenant, config, name, type):
        config_created = super().create_tenant_config(
            tenant, config, name=name, type=type
        )
        if config["mode"] == "server":
            server_host = get_app_config().get_host()
            user_url = server_host + reverse(
                f'api:{self.pname}_tenant:{self.pname}_scim_users',
                args=[tenant.id, config_created.id],
            )
            group_url = server_host + reverse(
                f'api:{self.pname}_tenant:{self.pname}_scim_groups',
                args=[tenant.id, config_created.id],
            )
            config["group_url"] = group_url
            config["user_url"] = user_url
            # 生成用于认证的token和secret
            secret = uuid.uuid4().hex
            config["secret"] = secret
            body = {"sub": config_created.id.hex}
            config["token"] = jwt.encode(body, secret, algorithm="HS256")
            config_created.config = config
            config_created.save()
        return config_created

    @abstractmethod
    def create_user(self, request, resource, correlation_identifier):
        """
        抽象方法
        Args:
            request (HttpRequest): Django 请求
            resource (scim_server.schemas.core2_enterprise_user.Core2EnterpriseUser): SCIM用户对象
            correlation_identifier (str): 请求唯一标识
        """
        raise NotImplementedException()

    @abstractmethod
    def create_group(self, request, resource, correlation_identifier):
        """
        抽象方法
        Args:
            request (HttpRequest): Django 请求
            resource (scim_server.schemas.core2_group.Core2Group): SCIM组织对象
            correlation_identifier (str): 请求唯一标识
        """
        raise NotImplementedException()

    @abstractmethod
    def delete_user(self, request, resource_identifier, correlation_identifier):
        """
        抽象方法
        Args:
            request (HttpRequest): Django 请求
            resource_identifier (str): 用户ID
            correlation_identifier (str): 请求唯一标识
        """
        raise NotImplementedException()

    @abstractmethod
    def delete_group(self, request, resource_identifier, correlation_identifier):
        """
        抽象方法
        Args:
            request (HttpRequest): Django 请求
            resource_identifier (str): 组织ID
            correlation_identifier (str): 请求唯一标识
        """
        raise NotImplementedException()

    @abstractmethod
    def replace_user(self, request, resource, correlation_identifier):
        """
        抽象方法
        Args:
            request (HttpRequest): Django 请求
            resource (scim_server.schemas.core2_enterprise_user.Core2EnterpriseUser): SCIM用户对象
            correlation_identifier (str): 请求唯一标识
        """
        raise NotImplementedException()

    @abstractmethod
    def replace_group(self, request, resource, correlation_identifier):
        """
        抽象方法
        Args:
            request (HttpRequest): Django 请求
            resource (scim_server.schemas.core2_group.Core2Group): SCIM组织对象
            correlation_identifier (str): 请求唯一标识
        """
        raise NotImplementedException()

    @abstractmethod
    def retrieve_user(self, request, parameters, correlation_identifier):
        """
        抽象方法
        Args:
            request (HttpRequest): Django 请求
            parameters (scim_server.protocol.resource_retrieval_parameters.ResourceRetrievalParamters): Retrieve请求对象
            correlation_identifier (str): 请求唯一标识
        """
        raise NotImplementedException()

    @abstractmethod
    def retrieve_group(self, request, parameters, correlation_identifier):
        """
        抽象方法
        Args:
            request (HttpRequest): Django 请求
            parameters (scim_server.protocol.resource_retrieval_parameters.ResourceRetrievalParamters): Retrieve请求对象
            correlation_identifier (str): 请求唯一标识
        """
        raise NotImplementedException()

    @abstractmethod
    def update_user(self, request, patch, correlation_identifier):
        """
        抽象方法
        Args:
            request (HttpRequest): Django 请求
            patch (scim_server.service.patch.Patch): Patch参数对象
            correlation_identifier (str): 请求唯一标识
        """
        raise NotImplementedException()

    @abstractmethod
    def update_group(self, request, patch, correlation_identifier):
        """
        抽象方法
        Args:
            request (HttpRequest): Django 请求
            patch (scim_server.service.patch.Patch): Patch参数对象
            correlation_identifier (str): 请求唯一标识
        """
        raise NotImplementedException()

    @abstractmethod
    def query_users(self, request, parameters, correlation_identifier):
        """
        抽象方法
        Args:
            request (HttpRequest): Django 请求
            parameters (scim_server.protocol.query_parameters.QueryParameters): Query请求对象
            correlation_identifier (str): 请求唯一标识
        Returns:
            List[Core2EnterpriseUser]: 返回scim_server模块中的标准用户对象列表
        """
        pass

    @abstractmethod
    def query_groups(self, request, parameters, correlation_identifier):
        """
        抽象方法
        Args:
            request (HttpRequest): Django 请求
            parameters (scim_server.protocol.query_parameters.QueryParameters): Query请求对象
            correlation_identifier (str): 请求唯一标识
        Returns:
            List[Core2Group]: 返回scim_server模块中的标准组织对象列表
        """
        pass

composite_model (BaseModel) django-model

TenantExtensionConfig(id, is_del, is_active, updated, created, tenant, extension, config, name, type)

Source code in arkid/core/extension/scim_sync.py

class TenantExtensionConfig(BaseModel):

    class Meta(object):
        verbose_name = _("插件运行时配置")
        verbose_name_plural = _("插件运行时配置")

    tenant = models.ForeignKey('core.Tenant', blank=False, on_delete=models.PROTECT, verbose_name=_('租户'))
    extension = models.ForeignKey('Extension', blank=False, on_delete=models.PROTECT, verbose_name=_('插件'))
    config = models.JSONField(blank=True, default=dict, verbose_name=_('Runtime Config','运行时配置'))
    name = models.CharField(max_length=128, default='', verbose_name=_('名称'))
    type = models.CharField(max_length=128, default='', verbose_name=_('类型'))

config: JSONField blank django-field

Runtime Config

created: DateTimeField blank django-field nullable

创建时间

extension: ForeignKey django-field

插件

id: UUIDField django-field

ID

is_active: BooleanField django-field

是否可用

is_del: BooleanField django-field

是否删除

name: CharField django-field

名称

tenant: ForeignKey django-field

租户

type: CharField django-field

类型

updated: DateTimeField blank django-field nullable

更新时间

create_group(self, request, resource, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
resource	scim_server.schemas.core2_group.Core2Group	SCIM组织对象	required
correlation_identifier	str	请求唯一标识	required

Source code in arkid/core/extension/scim_sync.py

@abstractmethod
def create_group(self, request, resource, correlation_identifier):
    """
    抽象方法
    Args:
        request (HttpRequest): Django 请求
        resource (scim_server.schemas.core2_group.Core2Group): SCIM组织对象
        correlation_identifier (str): 请求唯一标识
    """
    raise NotImplementedException()

create_tenant_config(self, tenant, config, name, type)

创建运行时配置

Parameters:

Name	Type	Description	Default
tenant	Tenant	租户	required
config	dict	config	required
name	str	运行时配置名字	required
type	str	配置类型	required

Returns:

Type	Description
TenantExtensionConfig	创建的对象

Source code in arkid/core/extension/scim_sync.py

def create_tenant_config(self, tenant, config, name, type):
    config_created = super().create_tenant_config(
        tenant, config, name=name, type=type
    )
    if config["mode"] == "server":
        server_host = get_app_config().get_host()
        user_url = server_host + reverse(
            f'api:{self.pname}_tenant:{self.pname}_scim_users',
            args=[tenant.id, config_created.id],
        )
        group_url = server_host + reverse(
            f'api:{self.pname}_tenant:{self.pname}_scim_groups',
            args=[tenant.id, config_created.id],
        )
        config["group_url"] = group_url
        config["user_url"] = user_url
        # 生成用于认证的token和secret
        secret = uuid.uuid4().hex
        config["secret"] = secret
        body = {"sub": config_created.id.hex}
        config["token"] = jwt.encode(body, secret, algorithm="HS256")
        config_created.config = config
        config_created.save()
    return config_created

create_user(self, request, resource, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
resource	scim_server.schemas.core2_enterprise_user.Core2EnterpriseUser	SCIM用户对象	required
correlation_identifier	str	请求唯一标识	required

Source code in arkid/core/extension/scim_sync.py

@abstractmethod
def create_user(self, request, resource, correlation_identifier):
    """
    抽象方法
    Args:
        request (HttpRequest): Django 请求
        resource (scim_server.schemas.core2_enterprise_user.Core2EnterpriseUser): SCIM用户对象
        correlation_identifier (str): 请求唯一标识
    """
    raise NotImplementedException()

delete_group(self, request, resource_identifier, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
resource_identifier	str	组织ID	required
correlation_identifier	str	请求唯一标识	required

Source code in arkid/core/extension/scim_sync.py

@abstractmethod
def delete_group(self, request, resource_identifier, correlation_identifier):
    """
    抽象方法
    Args:
        request (HttpRequest): Django 请求
        resource_identifier (str): 组织ID
        correlation_identifier (str): 请求唯一标识
    """
    raise NotImplementedException()

delete_user(self, request, resource_identifier, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
resource_identifier	str	用户ID	required
correlation_identifier	str	请求唯一标识	required

Source code in arkid/core/extension/scim_sync.py

@abstractmethod
def delete_user(self, request, resource_identifier, correlation_identifier):
    """
    抽象方法
    Args:
        request (HttpRequest): Django 请求
        resource_identifier (str): 用户ID
        correlation_identifier (str): 请求唯一标识
    """
    raise NotImplementedException()

get_groups_users(self, config)

Parameters:

Name	Type	Description	Default
config	arkid.extension.models.TenantExtensionConfig	Client模式创建的配置	required

Source code in arkid/core/extension/scim_sync.py

def get_groups_users(self, config):
    """
    Args:
        config (arkid.extension.models.TenantExtensionConfig): Client模式创建的配置
    """
    sync_server_id = config.config.get("sync_server", {}).get("id")
    server_config = TenantExtensionConfig.active_objects.filter(
        id=sync_server_id
    ).first()
    if not server_config:
        logger.error(f"No scim sync server config found: {sync_server_id}")
        return None, None
    group_url = server_config.config["group_url"]
    user_url = server_config.config["user_url"]
    token = server_config.config["token"]
    groups = self.get_data(group_url, token).get("Resources")
    users = self.get_data(user_url, token).get("Resources")
    return groups, users

load(self)

抽象方法，插件加载的入口方法

Source code in arkid/core/extension/scim_sync.py

def load(self):
    class UsersView(UsersViewTemplate):
        @property
        def provider(this):
            return self

        @method_decorator(jwt_token_required)
        def dispatch(self, request, *args, **kwargs):
            return super().dispatch(request, *args, **kwargs)

    class GroupsView(GroupsViewTemplate):
        @property
        def provider(this):
            return self

        @method_decorator(jwt_token_required)
        def dispatch(self, request, *args, **kwargs):
            return super().dispatch(request, *args, **kwargs)

    scim_server_urls = [
        re_path(
            rf'^scim/{self.pname}/(?P<config_id>[\w-]+)/Users(?:/(?P<uuid>[^/]+))?$',
            UsersView.as_view(),
            name=f'{self.pname}_scim_users',
        ),
        # re_path(r'^Groups/.search$', views.GroupSearchView.as_view(), name='groups-search'),
        re_path(
            rf'^scim/{self.pname}/(?P<config_id>[\w-]+)/Groups(?:/(?P<uuid>[^/]+))?$',
            GroupsView.as_view(),
            name=f'{self.pname}_scim_groups',
        ),
    ]
    self.register_routers(scim_server_urls, True)
    super().load()

query_groups(self, request, parameters, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
parameters	scim_server.protocol.query_parameters.QueryParameters	Query请求对象	required
correlation_identifier	str	请求唯一标识	required

Returns:

Type	Description
List[Core2Group]	返回scim_server模块中的标准组织对象列表

Source code in arkid/core/extension/scim_sync.py

@abstractmethod
def query_groups(self, request, parameters, correlation_identifier):
    """
    抽象方法
    Args:
        request (HttpRequest): Django 请求
        parameters (scim_server.protocol.query_parameters.QueryParameters): Query请求对象
        correlation_identifier (str): 请求唯一标识
    Returns:
        List[Core2Group]: 返回scim_server模块中的标准组织对象列表
    """
    pass

query_users(self, request, parameters, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
parameters	scim_server.protocol.query_parameters.QueryParameters	Query请求对象	required
correlation_identifier	str	请求唯一标识	required

Returns:

Type	Description
List[Core2EnterpriseUser]	返回scim_server模块中的标准用户对象列表

Source code in arkid/core/extension/scim_sync.py

@abstractmethod
def query_users(self, request, parameters, correlation_identifier):
    """
    抽象方法
    Args:
        request (HttpRequest): Django 请求
        parameters (scim_server.protocol.query_parameters.QueryParameters): Query请求对象
        correlation_identifier (str): 请求唯一标识
    Returns:
        List[Core2EnterpriseUser]: 返回scim_server模块中的标准用户对象列表
    """
    pass

replace_group(self, request, resource, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
resource	scim_server.schemas.core2_group.Core2Group	SCIM组织对象	required
correlation_identifier	str	请求唯一标识	required

Source code in arkid/core/extension/scim_sync.py

@abstractmethod
def replace_group(self, request, resource, correlation_identifier):
    """
    抽象方法
    Args:
        request (HttpRequest): Django 请求
        resource (scim_server.schemas.core2_group.Core2Group): SCIM组织对象
        correlation_identifier (str): 请求唯一标识
    """
    raise NotImplementedException()

replace_user(self, request, resource, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
resource	scim_server.schemas.core2_enterprise_user.Core2EnterpriseUser	SCIM用户对象	required
correlation_identifier	str	请求唯一标识	required

Source code in arkid/core/extension/scim_sync.py

@abstractmethod
def replace_user(self, request, resource, correlation_identifier):
    """
    抽象方法
    Args:
        request (HttpRequest): Django 请求
        resource (scim_server.schemas.core2_enterprise_user.Core2EnterpriseUser): SCIM用户对象
        correlation_identifier (str): 请求唯一标识
    """
    raise NotImplementedException()

retrieve_group(self, request, parameters, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
parameters	scim_server.protocol.resource_retrieval_parameters.ResourceRetrievalParamters	Retrieve请求对象	required
correlation_identifier	str	请求唯一标识	required

Source code in arkid/core/extension/scim_sync.py

@abstractmethod
def retrieve_group(self, request, parameters, correlation_identifier):
    """
    抽象方法
    Args:
        request (HttpRequest): Django 请求
        parameters (scim_server.protocol.resource_retrieval_parameters.ResourceRetrievalParamters): Retrieve请求对象
        correlation_identifier (str): 请求唯一标识
    """
    raise NotImplementedException()

retrieve_user(self, request, parameters, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
parameters	scim_server.protocol.resource_retrieval_parameters.ResourceRetrievalParamters	Retrieve请求对象	required
correlation_identifier	str	请求唯一标识	required

Source code in arkid/core/extension/scim_sync.py

@abstractmethod
def retrieve_user(self, request, parameters, correlation_identifier):
    """
    抽象方法
    Args:
        request (HttpRequest): Django 请求
        parameters (scim_server.protocol.resource_retrieval_parameters.ResourceRetrievalParamters): Retrieve请求对象
        correlation_identifier (str): 请求唯一标识
    """
    raise NotImplementedException()

sync(self, config, sync_log)

Parameters:

Name	Type	Description	Default
config	arkid.extension.models.TenantExtensionConfig	Client模式创建的配置	required

Source code in arkid/core/extension/scim_sync.py

def sync(self, config, sync_log):
    """
    Args:
        config (arkid.extension.models.TenantExtensionConfig): Client模式创建的配置
    """
    logger.info(
        f"============= Sync Start With Config: {config}/{config.config} ================"
    )
    groups, users = self.get_groups_users(config)
    if not groups or not users:
        return
    self.sync_groups(groups, config, sync_log)
    self.sync_users(users, config, sync_log)

sync_groups(self, groups, config, sync_log)

抽象方法

Parameters:

Name	Type	Description	Default
groups	List	SCIM Server返回的组织列表	required
config	arkid.extension.models.TenantExtensionConfig	Client模式创建的配置	required

Source code in arkid/core/extension/scim_sync.py

@abstractmethod
def sync_groups(self, groups, config, sync_log):
    """
    抽象方法
    Args:
        groups (List): SCIM Server返回的组织列表
        config (arkid.extension.models.TenantExtensionConfig): Client模式创建的配置
    """
    pass

sync_users(self, users, config, sync_log)

抽象方法

Parameters:

Name	Type	Description	Default
users	List	SCIM Server返回的用户列表	required
config	arkid.extension.models.TenantExtensionConfig	Client模式创建的配置	required

Source code in arkid/core/extension/scim_sync.py

@abstractmethod
def sync_users(self, users, config, sync_log):
    """
    抽象方法
    Args:
        users (List): SCIM Server返回的用户列表
        config (arkid.extension.models.TenantExtensionConfig): Client模式创建的配置
    """
    pass

update_group(self, request, patch, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
patch	scim_server.service.patch.Patch	Patch参数对象	required
correlation_identifier	str	请求唯一标识	required

Source code in arkid/core/extension/scim_sync.py

@abstractmethod
def update_group(self, request, patch, correlation_identifier):
    """
    抽象方法
    Args:
        request (HttpRequest): Django 请求
        patch (scim_server.service.patch.Patch): Patch参数对象
        correlation_identifier (str): 请求唯一标识
    """
    raise NotImplementedException()

update_user(self, request, patch, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
patch	scim_server.service.patch.Patch	Patch参数对象	required
correlation_identifier	str	请求唯一标识	required

Source code in arkid/core/extension/scim_sync.py

@abstractmethod
def update_user(self, request, patch, correlation_identifier):
    """
    抽象方法
    Args:
        request (HttpRequest): Django 请求
        patch (scim_server.service.patch.Patch): Patch参数对象
        correlation_identifier (str): 请求唯一标识
    """
    raise NotImplementedException()

Exemplary

extension_root.com_longgui_scim_sync_arkid.ScimSyncArkIDExtension (ScimSyncExtension)

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

class ScimSyncArkIDExtension(ScimSyncExtension):
    def load(self):
        self.register_scim_sync_schema('ArkID', ClientConfig, ServerConfig)
        super().load()

    def _get_arkid_user_attrs(self, user):
        active = user.get("active")
        if active is None:
            active = True

        return {
            "username": user.get("userName", ""),
            "is_active": active,
            "is_del": False,
        }

    def _get_arkid_user(self, scim_user, tenant, sync_log):
        scim_external_id = scim_user["id"]
        username = scim_user["userName"]
        arkid_user_attrs = self._get_arkid_user_attrs(scim_user)
        user_lookup = {
            "scim_external_id": scim_external_id,
            "username": username,
            "tenant": tenant,
        }
        # arkid_user, _ = User.objects.update_or_create(
        #     defaults=arkid_user_attrs, **user_lookup
        # )
        arkid_user = User.objects.filter(**user_lookup).first()
        if not arkid_user:
            user_lookup.update(arkid_user_attrs)
            arkid_user = User.objects.create(**user_lookup)
            sync_log.users_created += 1
        tenant.users.add(arkid_user)

        # 更新arkid_user所属的group
        arkid_user.usergroup_set.clear()
        for scim_group in scim_user.get("groups", []):
            scim_group_id = scim_group.get("value")
            arkid_group = self.scim_arkid_group_map.get(scim_group_id)
            if arkid_group:
                arkid_user.usergroup_set.add(arkid_group)
        # arkid_user.save()
        return arkid_user

    def _get_arkid_group(self, group, scim_arkid_map, tenant, sync_log):
        scim_external_id = group["id"] if "id" in group else group["value"]
        if scim_external_id not in scim_arkid_map:
            group_lookup = {"scim_external_id": scim_external_id, "tenant": tenant}
            arkid_group = UserGroup.objects.filter(**group_lookup).first()
            if not arkid_group:
                arkid_group = UserGroup.objects.create(**group_lookup)
                sync_log.groups_created += 1
            else:
                arkid_group.is_del = False
                arkid_group.is_active = True
            scim_arkid_map[scim_external_id] = arkid_group
            return arkid_group
        else:
            return scim_arkid_map[scim_external_id]

    def _sync_group_attr(self, arkid_group, scim_group):
        arkid_group.name = scim_group.get("displayName")
        arkid_group.save()

    def delete_group_from_root(self, root):
        logger.info(f"Delete Group {root.name} Start")
        children = root.children.all()
        if not children:
            root.delete()
            logger.info(f"delete group {root.name} success")
            return
        for item in children:
            self.delete_group_from_root(item)
        root.delete()
        logger.info(f"delete group {root.name} success")

    def sync_groups(self, groups, config, sync_log):
        """
        遍历groups中的SCIM 组织，逐一和ArkID中的组织匹配，如果不存在就创建，存在则更新，在此过程中
        同时遍历每个SCIM 组织中的members，同样的方式在ArkID中创建或更新组织，并且维护组织之间的父子关系，
        最后删除以前同步到ArkID但不在本次同步数据中的组织
        Args:
            groups (List): SCIM Server返回的组织列表
            config (arkid.extension.models.TenantExtensionConfig): Client模式创建的配置
        """
        logger.info("###### update&create groups ######")
        tenant = config.tenant
        self.scim_arkid_group_map = {}
        for group in groups:
            parent_group = self._get_arkid_group(
                group, self.scim_arkid_group_map, tenant, sync_log
            )
            self._sync_group_attr(parent_group, group)
            for member in group.get("members", []):
                sub_group = self._get_arkid_group(
                    member, self.scim_arkid_group_map, tenant, sync_log
                )
                sub_group.parent = parent_group

        logger.info("###### delete groups ######")
        groups_need_delete = (
            UserGroup.valid_objects.filter(tenant=config.tenant)
            .exclude(scim_external_id=None)
            .exclude(scim_external_id__in=self.scim_arkid_group_map.keys())
        )
        logger.info(f"******* groups to be deleted: {groups_need_delete} ********")
        root_groups = []
        for grp in groups_need_delete:
            if (grp.parent is None) or (grp.parent not in groups_need_delete):
                root_groups.append(grp)
        for root in root_groups:
            self.delete_group_from_root(root)
        delete_count = len(groups_need_delete)
        # groups_need_delete.delete()
        sync_log.groups_deleted = delete_count

    def sync_users(self, users, config, sync_log):
        """
        遍历users中的SCIM 用户记录，逐一和ArkID中的用户匹配，如果不存在匹配的就创建，存在则更新，
        最后删除以前同步到ArkID但不在本次同步数据中的用户
        Args:
            users (List): SCIM Server返回的用户列表
            config (arkid.extension.models.TenantExtensionConfig): Client模式创建的配置
        """
        logger.info("###### update&create users ######")
        tenant = config.tenant
        scim_user_ids = []
        for user in users:
            scim_user_ids.append(user["id"])
            try:
                arkid_user = self._get_arkid_user(user, tenant, sync_log)
            except IntegrityError as e:
                logger.error(e)
                logger.error(f"sync user failed: {user}")

        logger.info("###### delete users ######")
        users_need_delete = (
            tenant.users.filter(is_del=False)
            .exclude(scim_external_id=None)
            .exclude(scim_external_id__in=scim_user_ids)
        )
        logger.info(f"***** users to be deleted: {users_need_delete} ******")
        for u in users_need_delete:
            u.usergroup_set.clear()
            u.delete()
            sync_log.users_deleted += 1
            # users_need_delete.delete()

    def _get_scim_user(self, arkid_user):
        attr_map = {"id": "id", "username": "userName", "is_active": "active"}
        scim_user = Core2EnterpriseUser(userName='', groups=[])
        for arkid_attr, scim_attr in attr_map.items():
            value = getattr(arkid_user, arkid_attr)
            scim_path = Path.create(scim_attr)
            if (
                scim_path.schema_identifier
                and scim_path.schema_identifier == SchemaIdentifiers.Core2EnterpriseUser
            ):
                compose_enterprise_extension(scim_user, scim_path, value)
            else:
                compose_core2_user(scim_user, scim_path, value)

        # 生成用户所在的组
        parent_groups = arkid_user.usergroup_set.filter(is_del=0)
        for grp in parent_groups:
            scim_group = ScimUserGroup()
            scim_group.value = grp.id
            scim_group.display = grp.name
            scim_user.groups.append(scim_group)
        return scim_user

    def _get_scim_group(self, arkid_group):
        members = UserGroup.valid_objects.filter(parent=arkid_group)
        attr_map = {"id": "id", "name": "displayName"}
        scim_group = Core2Group(displayName='')
        for arkid_attr, scim_attr in attr_map.items():
            value = getattr(arkid_group, arkid_attr)
            scim_path = Path.create(scim_attr)
            compose_core2_group(scim_group, scim_path, value)
        for item in members:
            member = Member()
            member.value = item.id
            scim_group.members.append(member)
        return scim_group

    def _get_all_scim_users(self, tenant):
        scim_users = []
        arkid_users = User.valid_objects.filter(tenant=tenant)
        for arkid_user in arkid_users:
            scim_user = self._get_scim_user(arkid_user)
            scim_users.append(scim_user)
        return scim_users

    def _get_all_scim_groups(self, tenant):
        scim_groups = []
        arkid_groups = UserGroup.valid_objects.filter(tenant=tenant)
        for arkid_group in arkid_groups:
            scim_group = self._get_scim_group(arkid_group)
            scim_groups.append(scim_group)
        return scim_groups

    def query_users(self, request, parameters, correlation_identifier):
        """
        将ArkID中的用户转换成scim_server中的符合SCIM标准的Core2EnterpriseUser对象
        Args:
            request (HttpRequest): Django 请求
            parameters (scim_server.protocol.query_parameters.QueryParameters): Query请求对象
            correlation_identifier (str): 请求唯一标识
        Returns:
            List[Core2EnterpriseUser]: 返回scim_server模块中的标准用户对象列表
        """
        if not parameters.alternate_filters:
            all_users = self._get_all_scim_users(request.tenant)
            return all_users

    def query_groups(self, request, parameters, correlation_identifier):
        """
        将ArkID中的组织转换成scim_server中的符合SCIM标准的Core2Group对象
        Args:
            request (HttpRequest): Django 请求
            parameters (scim_server.protocol.query_parameters.QueryParameters): Query请求对象
            correlation_identifier (str): 请求唯一标识
        Returns:
            List[Core2Group]: 返回scim_server模块中的标准组织对象列表
        """
        if not parameters.alternate_filters:
            groups = self._get_all_scim_groups(request.tenant)
            return groups

    def create_user(self, request, resource, correlation_identifier):
        raise NotImplementedException()

    def create_group(self, request, resource, correlation_identifier):
        raise NotImplementedException()

    def delete_user(self, request, resource_identifier, correlation_identifier):
        raise NotImplementedException()

    def delete_group(self, request, resource_identifier, correlation_identifier):
        raise NotImplementedException()

    def replace_user(self, request, resource, correlation_identifier):
        raise NotImplementedException()

    def replace_group(self, request, resource, correlation_identifier):
        raise NotImplementedException()

    def retrieve_user(self, request, parameters, correlation_identifier):
        raise NotImplementedException()

    def retrieve_group(self, request, parameters, correlation_identifier):
        raise NotImplementedException()

    def update_user(self, request, patch, correlation_identifier):
        raise NotImplementedException()

    def update_group(self, request, patch, correlation_identifier):
        raise NotImplementedException()

create_group(self, request, resource, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
resource	scim_server.schemas.core2_group.Core2Group	SCIM组织对象	required
correlation_identifier	str	请求唯一标识	required

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

def create_group(self, request, resource, correlation_identifier):
    raise NotImplementedException()

create_user(self, request, resource, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
resource	scim_server.schemas.core2_enterprise_user.Core2EnterpriseUser	SCIM用户对象	required
correlation_identifier	str	请求唯一标识	required

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

def create_user(self, request, resource, correlation_identifier):
    raise NotImplementedException()

delete_group(self, request, resource_identifier, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
resource_identifier	str	组织ID	required
correlation_identifier	str	请求唯一标识	required

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

def delete_group(self, request, resource_identifier, correlation_identifier):
    raise NotImplementedException()

delete_user(self, request, resource_identifier, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
resource_identifier	str	用户ID	required
correlation_identifier	str	请求唯一标识	required

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

def delete_user(self, request, resource_identifier, correlation_identifier):
    raise NotImplementedException()

load(self)

抽象方法，插件加载的入口方法

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

def load(self):
    self.register_scim_sync_schema('ArkID', ClientConfig, ServerConfig)
    super().load()

query_groups(self, request, parameters, correlation_identifier)

将ArkID中的组织转换成scim_server中的符合SCIM标准的Core2Group对象

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
parameters	scim_server.protocol.query_parameters.QueryParameters	Query请求对象	required
correlation_identifier	str	请求唯一标识	required

Returns:

Type	Description
List[Core2Group]	返回scim_server模块中的标准组织对象列表

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

def query_groups(self, request, parameters, correlation_identifier):
    """
    将ArkID中的组织转换成scim_server中的符合SCIM标准的Core2Group对象
    Args:
        request (HttpRequest): Django 请求
        parameters (scim_server.protocol.query_parameters.QueryParameters): Query请求对象
        correlation_identifier (str): 请求唯一标识
    Returns:
        List[Core2Group]: 返回scim_server模块中的标准组织对象列表
    """
    if not parameters.alternate_filters:
        groups = self._get_all_scim_groups(request.tenant)
        return groups

query_users(self, request, parameters, correlation_identifier)

将ArkID中的用户转换成scim_server中的符合SCIM标准的Core2EnterpriseUser对象

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
parameters	scim_server.protocol.query_parameters.QueryParameters	Query请求对象	required
correlation_identifier	str	请求唯一标识	required

Returns:

Type	Description
List[Core2EnterpriseUser]	返回scim_server模块中的标准用户对象列表

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

def query_users(self, request, parameters, correlation_identifier):
    """
    将ArkID中的用户转换成scim_server中的符合SCIM标准的Core2EnterpriseUser对象
    Args:
        request (HttpRequest): Django 请求
        parameters (scim_server.protocol.query_parameters.QueryParameters): Query请求对象
        correlation_identifier (str): 请求唯一标识
    Returns:
        List[Core2EnterpriseUser]: 返回scim_server模块中的标准用户对象列表
    """
    if not parameters.alternate_filters:
        all_users = self._get_all_scim_users(request.tenant)
        return all_users

replace_group(self, request, resource, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
resource	scim_server.schemas.core2_group.Core2Group	SCIM组织对象	required
correlation_identifier	str	请求唯一标识	required

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

def replace_group(self, request, resource, correlation_identifier):
    raise NotImplementedException()

replace_user(self, request, resource, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
resource	scim_server.schemas.core2_enterprise_user.Core2EnterpriseUser	SCIM用户对象	required
correlation_identifier	str	请求唯一标识	required

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

def replace_user(self, request, resource, correlation_identifier):
    raise NotImplementedException()

retrieve_group(self, request, parameters, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
parameters	scim_server.protocol.resource_retrieval_parameters.ResourceRetrievalParamters	Retrieve请求对象	required
correlation_identifier	str	请求唯一标识	required

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

def retrieve_group(self, request, parameters, correlation_identifier):
    raise NotImplementedException()

retrieve_user(self, request, parameters, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
parameters	scim_server.protocol.resource_retrieval_parameters.ResourceRetrievalParamters	Retrieve请求对象	required
correlation_identifier	str	请求唯一标识	required

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

def retrieve_user(self, request, parameters, correlation_identifier):
    raise NotImplementedException()

sync_groups(self, groups, config, sync_log)

遍历groups中的SCIM 组织，逐一和ArkID中的组织匹配，如果不存在就创建，存在则更新，在此过程中 同时遍历每个SCIM 组织中的members，同样的方式在ArkID中创建或更新组织，并且维护组织之间的父子关系， 最后删除以前同步到ArkID但不在本次同步数据中的组织

Parameters:

Name	Type	Description	Default
groups	List	SCIM Server返回的组织列表	required
config	arkid.extension.models.TenantExtensionConfig	Client模式创建的配置	required

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

def sync_groups(self, groups, config, sync_log):
    """
    遍历groups中的SCIM 组织，逐一和ArkID中的组织匹配，如果不存在就创建，存在则更新，在此过程中
    同时遍历每个SCIM 组织中的members，同样的方式在ArkID中创建或更新组织，并且维护组织之间的父子关系，
    最后删除以前同步到ArkID但不在本次同步数据中的组织
    Args:
        groups (List): SCIM Server返回的组织列表
        config (arkid.extension.models.TenantExtensionConfig): Client模式创建的配置
    """
    logger.info("###### update&create groups ######")
    tenant = config.tenant
    self.scim_arkid_group_map = {}
    for group in groups:
        parent_group = self._get_arkid_group(
            group, self.scim_arkid_group_map, tenant, sync_log
        )
        self._sync_group_attr(parent_group, group)
        for member in group.get("members", []):
            sub_group = self._get_arkid_group(
                member, self.scim_arkid_group_map, tenant, sync_log
            )
            sub_group.parent = parent_group

    logger.info("###### delete groups ######")
    groups_need_delete = (
        UserGroup.valid_objects.filter(tenant=config.tenant)
        .exclude(scim_external_id=None)
        .exclude(scim_external_id__in=self.scim_arkid_group_map.keys())
    )
    logger.info(f"******* groups to be deleted: {groups_need_delete} ********")
    root_groups = []
    for grp in groups_need_delete:
        if (grp.parent is None) or (grp.parent not in groups_need_delete):
            root_groups.append(grp)
    for root in root_groups:
        self.delete_group_from_root(root)
    delete_count = len(groups_need_delete)
    # groups_need_delete.delete()
    sync_log.groups_deleted = delete_count

sync_users(self, users, config, sync_log)

遍历users中的SCIM 用户记录，逐一和ArkID中的用户匹配，如果不存在匹配的就创建，存在则更新， 最后删除以前同步到ArkID但不在本次同步数据中的用户

Parameters:

Name	Type	Description	Default
users	List	SCIM Server返回的用户列表	required
config	arkid.extension.models.TenantExtensionConfig	Client模式创建的配置	required

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

def sync_users(self, users, config, sync_log):
    """
    遍历users中的SCIM 用户记录，逐一和ArkID中的用户匹配，如果不存在匹配的就创建，存在则更新，
    最后删除以前同步到ArkID但不在本次同步数据中的用户
    Args:
        users (List): SCIM Server返回的用户列表
        config (arkid.extension.models.TenantExtensionConfig): Client模式创建的配置
    """
    logger.info("###### update&create users ######")
    tenant = config.tenant
    scim_user_ids = []
    for user in users:
        scim_user_ids.append(user["id"])
        try:
            arkid_user = self._get_arkid_user(user, tenant, sync_log)
        except IntegrityError as e:
            logger.error(e)
            logger.error(f"sync user failed: {user}")

    logger.info("###### delete users ######")
    users_need_delete = (
        tenant.users.filter(is_del=False)
        .exclude(scim_external_id=None)
        .exclude(scim_external_id__in=scim_user_ids)
    )
    logger.info(f"***** users to be deleted: {users_need_delete} ******")
    for u in users_need_delete:
        u.usergroup_set.clear()
        u.delete()
        sync_log.users_deleted += 1
        # users_need_delete.delete()

update_group(self, request, patch, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
patch	scim_server.service.patch.Patch	Patch参数对象	required
correlation_identifier	str	请求唯一标识	required

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

def update_group(self, request, patch, correlation_identifier):
    raise NotImplementedException()

update_user(self, request, patch, correlation_identifier)

抽象方法

Parameters:

Name	Type	Description	Default
request	HttpRequest	Django 请求	required
patch	scim_server.service.patch.Patch	Patch参数对象	required
correlation_identifier	str	请求唯一标识	required

Source code in extension_root/com_longgui_scim_sync_arkid/__init__.py

def update_user(self, request, patch, correlation_identifier):
    raise NotImplementedException()

Permanent data synchronization

评论